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TO  THE  EDUCATIONAL  RESOURCES 
INFORMATION  CENTER  (ERIC)  " 


PREFACE 


For  the  twelfth  year,  the  Research  and  Theory  Division  of  the  Association  for 
Educational  Communications  and  Technology  (AECT)  is  publishing  these 
Proceedings.  Papers  published  in  this  volume  were  presented  at  the  national  AECT 
Convention  in  Anaheim,  California.  A  limited  quantity  of  this  volume  were 
printed  and  sold.  It  is  also  available  on  microfiche  through  the  Educational 
Resources  Information  Clearinghouse  (ERIC)  system. 

REFEREEING  PROCESS:  All  research  papers  selected  for  presentation  at  the 
AECT  Convention  and  included  in  this  Proceedings  were  subjected  to  a  rigorous 
blind  reviewing  process.  Proposals  were  submitted  to  Steven  Ross  and  Gary 
Morrison  of  Memphis  State  University  or  Marina  Mclsaac  of  Arizona  State 
University.  All  references  to  author  were  removed  from  proposals  before  they  were 
:>ubmitted  to  referees  for  review.  Approximately  fifty  percent  of  the  manuscripts 
submitted  for  consideration  were  selected  for  presentation  at  the  Convention  and 
for  Publication  in  these  Proceedings.  The  papers  contained  in  this  document 
represent  some  of  the  most  current  thinking  in  educational  communications  and 
technology. 

This  volume  contains  two  indexes  covering  this  volume.  The  first  is  an 
author  index;  the  second  is  a  descriptor  index.  This  year  and  1989  are  included.  The 
two  indexes  will  be  updated  in  future  editions  of  this  Proceedings.  The  index  for 
volumes  1-6  (1979-84)  is  included  in  the  1986  Proceedings,  and  the  index  for 
volumes  7-10  is  in  the  1988  Proceedings. 

M.  R.  Simonson 
Editor 
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Association  for  Educational 
Commtinications  and  Technology 

WhatisAECT? 

AECT  is  the  only  national,  professional  associa- 
tion dedicated  to  the  improvement  of  instruction 
through  the  effective  use  of  media  and  technol- 
ogy. AECT  assists  itis  members  in  using  technol- 
ogy in  their  jobs  arid  to  ei\hance  the  learning 
process. 

Who  belongs  to  AECT? 

O  Media  Specialists 

O  Educators 

0  Instructional  Designers 

O  Learning  Resource  Specialists 

O  Curriculum  Developers 

O  Television  producers  and  directors 

O  Communications  specialists 

O  Education  Adnninistrators 

O  Others  who  req't»re  expertise  in  instructional 

technology 

AECT  History 

AECT  began  as  the  Department  of  Audio-Visual 
Instruction  at  the  National  Education  Associa- 
tion in  1923,  in  the  days  when  visual  aids  were 
blackboards  and  easels.  In  1947,  as  educators 
were  adapting  technology  used  to  train  World 
War  II  service  personnel  for  the  classroom,  the 
name  of  the  organization  became  the  Depart- 
ment of  Visual  Instruction  (DA  VI).  Twelve  years 
later,  DAVI  became  an  affiliate  of  the  NEA  and 
finally  the  autonomous  association,  AECT,  in 
1974. 


Today,  AECT  keeps  an  eye  o?i  the  future  of  in- 
structional technok  gy  while  assL.ting  educators 
with  the  changes  and  chaii<3nge3  that  face  them 
now.  AECT  members  are  professionals  devoted 
to  quality  education.  They  care  about  doing  their 
jobs  better  and  want  to  embrace  new  methods, 
new  equipment,  and  new  techniques  that  assist 
learning. 

AECr  Affiliates 

AECT  has  45  state  and  11  national  affiliates,  and 
has  recently  established  several  chapters  surround- 
ing major  universities  and  metropolitan  areas. 
Tliese  affiliated  organizations  add  a  locali7ed 
din«nsion  to  your  AECT  membership  and  allow 
for  more  interaction  among  your  colleagues.  For 
more  details  on  chapters  and  affiliates  in  your 
area,  contact  the  AECT  National  Office. 

AECT  National  Convention  and 
INFOCOMM  International  Exposition 

Each  year,  AECT  brings  top  speakers  to  exciting 
locations,  and  pre^nts  over  3CK/  sessions  and 
special  events  to  provide  the  best  training  avail- 
able in  the  use  of  media  in  education  and  instruc- 
tion. The  convention  features  the  IT^FOCOMM 
International  Exposition,  an  extravaganza  of  over 
300  companies  showing  the  latest  products  to  use 
in  educ*ation  and  training. 

Research  and  Theory  Division  (RTD)  improves 
the  design,  execution,  utilization,  evaluation,  and 
dissemination  of  audiovisual  communications 
research;  advises  educators  on  the  effective  utili- 
zations of  theresults  of  the  research;  and  promotes 
applied  and  theoretical  research  on  the  use  of 
media. 
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Learner  Control  of  Review  in  Computer  Assisted  Instruction  within  a 

Military  Training  Environment 

In  1988  the  United  States  Army  planned  on  spending  in  excess  of  1 .2 
billion  dolhirs  on  individual  training,  and  in  1989  that  budget  increased  to  1 .3 
billion  dollars  (Office  of  the  Director  of  the  Army,  1987).  Although  these  dollars 
represent  only  a  small  fraction  of  the  total  Army  budget,  they  indicate  that  the 
Army's  commitment  to  training  is  extensive.  In  the  face  of  Congressional  pressure 
to  limit  the  Defense  Budget  as  part  of  a  general  plan  to  bring  the  National  Debt 
under  control,  the  US  Army  must  make  every  effort  to  increase  training  efficiency 
and  effectiveness.  A  major  source  for  achieving  better  quality  training  has  been  the 
use  of  instructional  technology  which  Gagne'  (1987)  defines  as  practical  techniques 
of  instructional  delivery  that  systematically  aim  for  effective  learning. 

The  issue  of  efficiency  in  the  training  environment  has  a  long  tradition  in  the 
Amiy.  Many  of  the  developments  in  instructional  technology  used  in  the 
educational  environment  at  large  came  from  research  conducted  for  the  military 
since  World  War  I  (Noble,  1988).  The  use  of  computers  as  a  medium  of 
instructional  technology  has  been  a  major  contribution  to  the  developmental 
progression  of  this  technology.  Most  recently  the  Army  has  commenced  an  effort 
to  integrate  the  use  of  personal  computers  to  deliver  instruction  at  various  training 
institutions.  Invariably  the  success  of  the  Electronic  Information  Delivery  System 
(EIDS)  will  rest  on  the  quality  of  the  instructional  software,  officially  reierred  to  as 
Interactive  Courseware  (ICW)  (TRADOC,  1987). 

One  method  of  making  a  qualitative  change  in  ICW  is  improving  the  nature 
of  interactivity  between  the  student  and  the  computer.  A  feature  of  interactivity  that 
can  be  manipulated  by  the  instructional  designer  is  the  amount  of  learner-control 
afforded  by  the  courseware.  Over  the  years  the  term,  learner-control,  has  meant 
different  things  to  insuiictional  designers.  The  first  educational  use  of  the  temi 
learner-control  (Mager  &  Clark,  1963)  referred  to  the  student*s  ability  to  sequence 
the  content  of  a  particuhu-  course  of  inso-uction.  Later,  allowing  the  student  the 
ability  to  control  the  pace  of  instruction  came  under  the  heading  of  learner  control. 
Research  has  also  been  done  that  allows  the  learner  to  control  the  questions  and 
style  of  instruction  (McCann,  1981).  Because  the  term  learner  control  can  be  used 
in  reference  to  different  features  of  ICW,  a  descriptor  is  used  to  qualify  type  (e.g. 
learner-control  of  review). 

The  role  which  learner  control  plays  in  the  success  of  ICW  is  not  entirely 
clecU*.  Several  studies  have  suggested  that  learner  control  can  be  as  simple  as 
control  of  pace  and  be  very  successful  in  terms  of  achievement  (Park  &  lennyson, 
1983;  Ross  &  Rakow,  1981  among  others).  A  meta-analysis  by  Steinberg  (1977) 
indicates  that  external  cono-ol  of  instruction  by  the  computer  is  as  effective  as 
learner  control  and  has  the  added  advantage  of  taking  less  time  to  complete,  which 
has  implications  for  the  efficiency  of  insuiiction.  Other  studies  have  suggested  that 
giving  the  learner  more  cono-ol  over  the  instruction  has  produced  positive  results 
(Merrill,  1980;  Hannafin  &  Colamaio,  1987).  One  advantage  for  learner  control 
that  seems  to  be  consistent  is  that  students  show  a  better  attitude  toward  the 
instruction  when  they  have  greater  control,  which  in  turn  increases  motivation 
(Sawyer,  1985).  Because  of  these  inconsistencies.  Snow  (1980)  has  put  forth  the 
argument  that  the  conditions  of  effective  learner  conu-ol  still  merit  study.  The  result 
is  that  a  primary  emphasis  of  CAI  design  research  has  been  on  le:u-ner  control  of 
pace  and  content  sequence,  but  there  has  been  little  significant  research  on 
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providing  the  learner  with  the  ability  (control)  to  review  content  when  the  student 
feels  a  need  for  review. 

The  present  research  vv^as  prompted,  in  part,  by  design  constraints  noted  in 
two  related  research  projects  (Ho  et  al,  1986;  Kinzie  et  al,  1987).  In  the  study 
conducted  by  Ho  etal  (1986)  rhe  ICW  did  not  include  embedded  questions  as  a 
pan  of  the  instructional  design.  One  explanation  for  the  significance  the  review 
option  that  study  achieved,  was  possibly  due  to  the  absence  of  review  questions. 
The  Kinzie  etal.  (1987)  study  compared  program  and  learner  controlled  review  of 
embedded  questions  after  an  incorrect  response.  The  study  achieved  significance 
which  favored  the  use  of  learner  control  Yet,  that  research  leaves  open  the 
question  of  whether  a  review  is  better  thkn  not  having  a  review.  The  present  study 
addressed  both  issues. 

The  theoietical  base  for  the  research  questions  addressed  came  from  a 
review  of  past  CAi  research,  research  based  ICW  design  factors,  map  reading,  and 
military  training.  More  recently  several  studies  have  examined  different  design 
factors  including;  embedded  questions,  feedback,  learner  control,  screen  design, 
and  the  use  of  text.  The  design  component  investigated  in  the  present  project  was 
the  use  of  review;  which  are  those  activities  that  make  use  of  the  content  after  the 
original  learning  has  occurred  (Blair  et  al.,  1968).  The  research  used  to  support  the 
hypotheses  came  from  a  synthesis  of  findings  found  in  research  concerning 
traditional  teaching  techniques,  open-book  testing,  and  use  of  review  in  ICW,  The 
conclusion  was  that  there  is  sufficient  evider/  e  to  suggest  that  a  review  option 
could  improve  a  students  achievement  on  fact,  procedure,  and  problem  solving 
questions  without  adversely  effecting  efficiency. 

The  central  research  question  was:  "Does  providing  students  with  an  option 
to  review  content  in  ICW,  when  attempting  to  answer  embedded  questions, 
improve  their  achievement  on  a  posttest  without  adversely  effecting  the  amount  of 
time  it  takes  to  complete  the  instruction?"  The  courseware  made  use  of  fact, 
procedure,  and  problem  solving  questions,  which  represents  three  different  types  of 
learning  (Schaffer  &Hannafin,  1986), 

Independent  and  Dependent  Vrtriahles 

The  independent  variable  for  this  study  was  analyzed  for  effect  in  relation  to 
five  dependent  variables.  The  independent  variable  was  learner  control  of  a' 
Content  Review  Option  (CRO),  A  course  on  military  map  reading,  that  contained 
eight  lessons,  was  converted,  using  research  based  design  principles,  into  two 
versions  of  CAI  from  an  existing  programmed  text.  In  version  one.  Limited  Access 
to  the  Content  Review  Option  (LACRO)  (Fig.  1),  subjects  had  access  to  a 
summarized  review  of  a  lesson  s  content  after  the  lesson  had  been  completed  but 
before  encountering  the  embedded  questions.  In  version  two.  Open  Access  to  the 
Content  Review  Option  (OACRO)  (Fig,  2),  subjects  had  access  ;o  the  review  at  the 
end  of  a  lesson,  when  presented  a  question,  and  after  receiving  feedback  to  a 
question  response.  The  five  dependent  variables  were  achievement  [four  factors:  1) 
fact  questions;  2)  procedure  questions;  3)  problem  solving  questions,  and  4)  total 
score],  instructional  time,  attention  time,  review  time,  and  acquisition  rate. 

Instrument 

The  recall  posttest  was  a  21  item,  multiple  choice  test.  The  test  covered 
seven  lessons  found  in  the  ICW,  Each  sub-lesson  had  a  fact,  procedural,  and 
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problem  solving  type  question.  The  test  was  patterned  after  a  test  designed  by  ihc 
U.S.  Amiy.  The  test  was  reviewed  by  a  panel  of  three  independent  subject  mailer 
experts  (inter-rater  reliability  =  .95),  and  went  through  post-hoc  reliability  testing 

(a  =  .68).  The  folio  A'ing  criteria  were  used  foi  uie  design  of  each  question 
(Hannafin  &  Colania:o,  1987): 

Factual  -  Requires  the  literal,  verbatim  recall  of  infoniiation  presented 
during  the  lesson. 

Procedural  -  Requires  the  integration  and  transfer  of  the  lesson  s  factual 
information  to  a  series  of  related  tasks. 

Problem  Solving  -  Requires  the  application  of  facts  or  procedures  to  the 
solution  of  novel  problems. 

In  addition  to  the  posttest,  an  informal  post- treatment  questionnaire  was 
completed  by  eacli  subject.  The  questionnaire  had  seven  questions  relating  to  the 
subjects  use  of  the  CAI  and  review  option.  The  results  from  this  survey  are 
covered  later 

Subjects 

The  subjects  used  in  this  study  were  58  Army  ROTC  cadets  drawn  from  the 
MS  I  (freshman),  MS  II  (sophomore),  and  MS  III  (junior)  classes  at  the  University 
of  Kansas.  An  additional  12  subjects  were  drawn  from  the  undergraduate  teacher 
program  in  the  School  of  Education.  The  subjects  were  randomly  assigned  to 
groups  and  the  groups  were  randomly  assigned  to  treatments. 

Data  Treatment 

A  two-group  design  was  used  for  this  research.  The  dependent  variables 
for  achievement,  and  attention  time  were  analyzed  using  MANOVA  procedures. 
ANOVA  procedures  were  used  to  analyze  the  acquisition  rate  data.  ANOVA 
procedures  were  used  to  analyze  the  instructional  time  data,  iuid  ANCOVA 
procedures  were  used  to  analyze  the  effect  of  review  time  on  insuiictional  time.  The 
level  of  significance  was  set  at  .05. 

Results 

Hypothesis  One:  Posttest 

The  null  hypothesis  states  that  there  is  no  significant  difference  in 
achievement  between  the  groups  as  measured  by  the  achievement  posttest.  The 
score  recorded  has  a  possible  range  of  1  -  21  and  is  one  of  four  scores  recorded 
using  one  posttest.  The  sub-scores  for  fact,  procedure,  and  problem  solving 
questions  are  discussed  in  hypothesis  two. 

To  test  Hypothesis  One,  a  MANOVA  wau  done  on  the  posttest  data  using 
the  SPSSx  statistical  package  on  the  University  of  Kansas  VAX  computer  system. 
The  Hotellings  criterion  for  multivariate  analysis  yielded,  F(4,65)  =  .659,  ^  =  .623, 
which  was  not  significant.  Mean  scores,  standai'd  deviations,  and  sample  sizes  for 
the  two  groups  are  reported  in  Table  1.  The  results  of  univariate  F-Tests  are  shown 
in  Table  2.  Hypothesis  one  was  not  rejected. 
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Hypothesis  Two:  Fact,  Procedure,  and  Problem  Solving 

The  null  hypothesis  has  the  following  three  parts.  One,  there  is  no 
significant  difference  on  achievement  for  fact  type  questions  between  the  two 
groups.  Two,  there  is  no  significant  difference  on  achievement  for  procedure  type 
questions  between  the  two  groups.  Three,  theie  is  no  significant  difference  on 
achievement  for  problem  solving  type  questions  between  the  two  groups.  The 
MANOVA  returned,  £(4,65)  =  .659,  j2=.623,  which  was  not  significant. 
Descriptive  statistics  are  presented  in  Table  3,  and  inferential  statistics  are  included 
in  Table  4.  Hypothesis  Two  was  not  rejected. 

Hypothesis  Three:  Acquisition  Rate 

The  null  hypothesis  states  that  there  is  no  significant  difference  in 
acquisition  rate  between  the  two  groups.  Acquisition  rate  was  based  on  the  recall  to 
instructional  time  ratio  (total  points  on  posttest  divided  by  instructional  time  in 
minutes  and  seconds).  Acquisition  rate  was  a  time-based  indicator  of  learning 
efficiency  for  the  two  treatments  (Schaffer  &  Hannafin,  1986).  A  one-way 
ANOVA,  using  the  MYSTAT  (SYSTAT,  Inc.)  computer  package  on  a  Macintosh 
SE,  was  done  on  the  acquisition  rate  data.  The  computed  ANOVA  for  the 
difference  between  the  treatment  groups  was,  jE(l,  68)  =  3.455,  £=.067. 
Therefore,  hypothesis  three  was  not  rejected.  Means,  standard  deviations,  and 
number  of  cases  are  displayed  for  each  group  in  Table  5.  The  ANOVA  summary  is 
presented  in  Table  6. 

Hypothesis  Four:  Attention  Time 

The  null  hypothesis  stated  there  is  no  significant  difference  in  attention  time 
between  tlie  two  groups.  The  data  is  divided  into  ^ine  parts.  The  Hrst  eight  pans 
was  the  time  spent  on  each  lesson.  The  ninth  data  point  was  the  currulation  of  j/ans 
1  through  8.  MANOVA  used  to  analyze  these  data  yielded  the  follow. ng,  E(9,60) 
=  1.139,  jj=.351,  which  was  not  significant.  Hypothesis  four  was  not  rejected. 
Means,  standard  deviations,  and  cell  sizes  are  shown  in  Table  7.  Relevant 
univariate  F-tests  are  included  in  Table  8. 

Hypothesis  Five:  Instructions!  Tline 

The  fifth  hypothesis  states  that  there  is  no  significant  difference  in 
instructional  time  means  between  the  two  groups.  Instructional  time  was  the 
amount  of  ame  it  takes  a  learner  to  complete  the  entire  course  of  instruction. 
Measurement  commenced  when  the  first  instructional  screen  was  encountered  and 
terminated  when  the  last  screen  of  the  lesson  was  completed.  An  ANOVA  was 
performed  which  produced  a  significant  instructional  time  effect,  £(1, 68)  = 
4.309,  p=.042.  Therefore  Hypothesis  Five  was  rejected.  To  further  test  the 
probability  that  the  review  option  was  the  cause  for  the  significant  difference  in 
Hypothesis  Five  an  ANCOVA,  using  the  MYSTAT  statistical  package  on  a 
Macintosh  SE  computer,  was  done  using  the  review  time  data  as  the  covariate.  The 
computations  resulted  in,  E(l.  67)  =  .770,  £=.383,  which  was  not  significant. 
Means,  standard  deviations,  and  cell  size  are  reported  in  Table  9.  ANOVA,  and 
ANCOVA  summaries  are  shown  in  Tables  10  and  1 1  respectively.  The 
interpretation  is  that  the  OACRO  group  spent  significantly  more  time  using  the  map 
reading  CAI  and  that  this  increase  in  time  was  because  of  review  time. 
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Informal  Data 

After  the  subjects  took  the  posttest  they  were  asked  to  respond  to  seven 
questions.  The  responses  to  those  questions  are  described  in  the  following 
paragraphs. 

Question  1 ;  *'Werc  you  aware  of  having  the  option  to  review  the  course 
material?"  Responses  indicate  that  95.7%  of  all  the  subjects  knew  they  had  the 
option  io  review  the  material. 

Question  2;  "Did  you  know  when  you  could  use  the  review  option?"  A 
majority  of  respondents  (97%),  that  answered  positively  in  Question  1,  replied 
positively  to  question  2.  The  high  percentage  of  positive  responses  to  these  two 
questions  led  to  the  cor'^^'^sion  that  retention  of  the  null  hypotheses  was  not  because 
of  subject  ignorance  of  u.  j  content  review  option. 

Question  3;  "Do  you  think  you  would  use  the  review  more  often  if  you 
could  see  the  questions  before  deciding  if  you  wanted  to  review?"  This  question 
was  only  directed  to  those  subjects  in  the  group  that  had  limited  access  to  the 
review.  The  majority  of  subjects  (81.3%)  responded  "Yes"  to  this  question.  This 
response  compares  favorably  with  tlie  behavior  of  the  subjects  in  the  open  access 
group  since  91%  used  the  review  after  they  saw  the  questions. 

Question  4;  "What  type  of  question  did  you  use  the  review  on  most  often'^" 
This  question  was  directed  to  the  subjects  in  the  group  that  had  open  access  to  the 
review.  The  tally  indicates  that;  8.6%  did  not  use  the  review  when  the  question 
was  presented,  17. 1%  used  the  review  only  on  fact  type  questions,  2.9%  used  the 
review  only  for  procedure  type  questions,  1 1.4%  used  the  review  only  for  problem 
solving  questions,  57.1%  used  the  review  on  both  fact  and  procedure  type 
questions,  and  2.9%  used  the  review  for  all  the  types  of  questions.  Ip  general, 
students  accessed  the  review  most  often  for  questions  which  require  the  use  of  rote 
memory. 

Question  5;  "Do  you  think  you  would  use  the  review  more  often  if  the 
questions  were  fil!  in-the-blank,  or  short  answer?"  As  with  Question  4  this 
question  was  directed  at  only  the  subjects  in  the  open  access  group.  Tlie  majority 
(74.3%)  of  the  OACRO  subjects  responded  positively  to  this  quernon.  This 
response  rate  suggests  that  the  use  of  multiple-choice  questions  effected  a  subject's 
decision  to  use  the  review  option. 

Question  6;  "What  changes  in  training  conditions  doyru  believe  would 
cause  you  to  use  the  review  more  often?"  Two  conditions  emerged.  A  minority 
(21%)  of  all  tiie  subjects  indicated  that  if  they  were  requii-ed  to  spend  a  specified 
amount  of  time  in  the  training  session  -  which  exceeded  the  amount  of  time  it  would 
nonnally  take  to  complete  the  instruction  -  they  would  use  the  review.  A  majority 
(61%)  of  all  the  subjects  indicated  that  if 'a  minimum  standard  of  achievement  was 
established  for  passing  then  they  would  probably  use  the  review  more  often.  This 
indicates  that  the  conditions  under  which  the  research  was  conducted  influenced  a 
subject's  decision  to  use  the  review.  The  other  18%  of  the  subjects  gave  no  answer 
to  this  question. 

Question  7;  "Did  you  have  any  problems  using  the  courseware  on  the 
computer?"  A  majonty  (93%)  of  all  the  subjects  indicated  that  they  had  no 
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difficulty  using  the  CAI  medium  employed  in  the  present  research.  The  reseiirchcr 
concludes  that  the  user  interface  of  the  instructional  medium  did  not  adversely  effect 
achievement. 

Conclusions 

Regarding  achievement,  it  was  concluded  that  providing  a  content  review 
option  did  not  produce  a  significant  difference  between  the  groups.  This  was  also 
true  for  the  three  types  of  questions  used  to  make  up  the  posttest.  This  indicates 
that  the  effect  the  use  of  review  questions  had  on  student  achievement  was  not 
enhanced  when  a  content  review  option  is  added  to  the  instructional  design.  Since 
the  content  in  this  study  is  comparable  to  the  content  used  in  the  studies  conducted 
by  Ho  et  ai  (1986)  and  Kinzie  et  aL  (1987)  the  present  research  raises  questions 
regarding  generalizations  made  in  those  previously  mentioned  studies.  The  Ho  et 
al.  (1986)  study  declared  that  the  results  indicate  the  use  of  review,  in  the  form  of  a 
lesson  summary,  is  an  effective  instructional  design  strategy.  Since  that  study  did 
not  use  embedded  questions,  which  is  considered  a  ICW  design  imperative  (Alessi 
&  Trollip,  1985;  Wager  &  Wager,  1985;  Hannafm  &  Peck.  1988),  then  it  is 
reasonable  to  question  if  the  same  results  would  be  realized  if  embedded  questions 
were  used  in  conjunction  with  the  review  opaon.  Kinzie  et  al.  (1987)  recognized 
that  research  conducted  in  this  area  should  be  done  within  the  context  of  sound 
instructional  design.  The  review  option  designed  for  that  study  complimented  the 
use  of  embedded  questions  but,  the  review  could  only  be  accessed  after  an  incorrect 
answer,  which  may  be  a  design  limitation  rather  than  a  design  flaw.  On  the  ba,s's  of 
their  results  they  stated  that  leanier  control  of  review  was  a  practical  design  factor. 
Kinzie  et  al.  (1987),  however,  did  not  address  whether  or  not  review  plus 
embedded  questions  is  more  effective  than  embedded  questions  alone. 

A  comparison  of  the  results  of  these  two  research  efforts  suggests  that 
installing  a  content  review  option  in  addition  to  the  use  of  embedded  questions 
would  be  an  effective  ICW  design.  But  the  results  of  the  present  study  indicates  the 
positive  effects  noted  in  the  Ho  and  Kinzie  studies  are  not  additive.  The  effects  of 
review  may  be  absorbed  by  the  effect  that  the  use  of  embedded  questions  has  on 
student  achievement  regardless  of  the  type  of  question  used. 

Tlie  results  of  the  present  study  also  conti-adict  the  findings  regarding 
students  attention  to  content  found  in  the  Kcerature  on  open  book  testing.  The 
research  reviewed  (Kalish,  1958;  Krarup  et  ai,  1974;  Weber  et  al.,  1983; 
Boniface,  1985)  consistently  found  that  students  facing  an  open  book  testing 
situation  spent  less  time  studying  the  content  to  be  tested  than  students  in  a  closed 
book  testing  situation.  In  this  study  there  was  no  significant  difference  between  the 
attention  time  for  the  LACRO  (M  =  47.49  min.)  and  the  OACRO  (M  =  54.96  min.) 
groups.  One  possible  explanation  is  that  when  students  use  CAI,  they  become 
absorbed  by  the  methodical  process  of  going  through  the  instruction.  Also, 
students  are  usually  in  an  environment  conducive  to  study  (i.e.,  computer  lab,  or 
learning  center)  whereas,  studying  text  allows  the  student  to  quickly  skim  material 
or  not  read  the  content.  Unless  students  make  a  conscious  effort  to  put  themselves 
in  an  environment  conducive  for  study  the  potential  for  innumerable  distractions 
(i.e.  TV,  radio,  other  people)  exists. 

Acquisition  rate  was  not  significant.  This  fact  supports  the  conclusions 
drawn  from  the  data  on  the  posttest.  The  review  option  does  not  increase  or 
decrease  a  student's  ability  to  learn  from  the  CAI.  Since  the  review  option  does  not 
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hinder  or  help  acquisition  the  designer  must  draw  on  other  infomuition  to  make  a 
design  choice. 

These  conclusions  do  little  to  address  tlie  highly  vocal  critics  of  CAI  that 
regularly  cite  the  linear  function  of  many  ICW  programs  as  being  limited  or  too 
simplistic  to  address  the  needs  of  learners  (Bork,  1982;  Pogrow,  1988;  Streibel, 
1986).  Instead  of  concluding  thai  the  present  study  has  failed  to  meet  the  demands 
of  the  critics,  I  would  suggest  that  the  critics  are  overstating  their  case.  It  may  be 
that  learners  do  not  employ  their  entire  repertoire  of  cognitive  skills  in  every 
learning  situation.  The  result  is  that  for  certain  content  or  learning  situations  a  linear 
approach  may  be  the  optimum  way  to  communicate  the  instruction  to  the  student; 
other  studies  suggest  that  such  is  the  case  (Hilgard,  Atkinson  &  Atkinson,  1975). 

Limitations 

A  partial  explanation  for  the  lack  of  significance  in  the  results  of  this 
research  can  be  explained  by  noted  limitations.  Six  areas  have  been  identified  as 
potentially  effecting  the  results  of  this  study.  These  items  include;  length  o£the 
instruction,  subject  motivation,  embedded  and  posttest  question  style,  type  of 
problem  solving,  learner  characteristics,  and  student  use  of  learner  control. 

The  instruction  ised  in  the  present  study  was  much  longer  than  the 
instruction  used  in  similar  studies.  Subjects  spent  an  average  of  78  minutes  in  just 
the  CAI  portion  of  the  treatment.  This  exceeds  normally  accepted  standards  for  the 
amount  of  time,  about  50  minutes,  an  adult  learner  is  expected  to  maintain  attention 
on  content  without  a  break.  As  a  result,  subjects  demonstrated  signs  of  fatigue 
(i.e.,  poor  posture,  yawning)  during  the  experiment  which  could  have  had  an 
adverse  effect  on  learning.  This  effect  however,  should  have  been  consistent 
across  the  treatments  since  the  amount  of  instruction  was  the  same  for  both  groups. 

Since  there  were  no  external  incentives  used  in  the  experiment,  motivation 
to  muster  the  content  was  determined  by  ihe  subject's  internal  desire  to  learn  map 
reading.  This  may  have  adversely  effected  learning,  ^  r  the  use  of  review,  as 
indicated  in  the  following  posttest  question:  "What  changes  in  training  conditions 
do  you  believe  would  cause  you  to  use  the  review  more  often?".  The  result  was 
that  61%  (43/70  S's)  of  the  subjects  indicated  that  establishing  a  required  minimum 
score  woulo  have  caused  them  to  use  the  content  review  option  more  often,  with 
another  21%  (15/70  S*s)  indicating  that  a  required  period  of  time  (that  exceeds  the 
actual  amount  of  time  to  complete  the  instruction)  for  studying  the  instruction  would 
cause  more  extensive  use  of  the  CRO.  Therefore,  it  seems  reasonable  to  assume 
that  motivation  had  an  effect  on  the  use  of  review,  which  may  have  ultimately  had 
an  effect  on  achievement. 

The  use  of  multiple  choice  questions  was  also  a  limiting  factor  which  may 
have  adversely  effected  the  use  of  review.  Multiple  choice  questions  do  not  require 
pure  recall  by  the  subject  since  the  possible  answers  provide  cues  that  aid  recall. 
Also,  the  subjects  j^dmitted  to  a  willingness  to  guess  at  the  correct  answer,  before 
using  the  review,  since  there  was  no  penalty  for  an  incorrect  response  to  a  question. 
Most  of  the  subjects  in  the  OACRO  group  (74%,  26/35  S's)  indicated  on  the  post- 
treatment  questionnaire  that  the  use  of  fill-in-the-blank,  or  short  answer  questions 
would  probably  have  caused  them  to  use  the  review  more  often. 

Another  limiting  factor  that  may  have  affected  the  use  of  review  was  the 
type  of  problem  solving  presented  in  the  ICW.  Specifically,  problem  solving  was 
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limited  to  algorithms.  This  type  of  problem  solving  is  a  given  procedure  that  will 
result  in  a  successful  solution  when  it  is  consistently  followed  (Ingram,  1988). 
Doing  an  intersection  on  a  map  is  an  example  of  algorithmic  problem  solving. 
Algorithmic  problem  solving  is  on  the  low  end  of  a  continuum  with  heuristics 
representing  the  opposite  end.  A  heuristic  is  a  general  strategy  that  can  be  used  to 
approach  a  problem  in  sach  a  way  that  it  will  not  ensure  success  but  will  increase 
the  probability  of  arriving  at  an  acceptable  solution.  By  their  very  nature  heuristics 
are  difficult  to  teach  no  matter  what  the  method  (Ingram,  1988).  This  is  because 
they  can  involve  the  use  of  complex  constructs  and  higher  order  rules  with 
inconsistent  applications  (i.e.,  rules  of  spelling  for  English). 

Because  all  soldiers,  regardless  of  ability,  are  expected  to  be  able  to  use  a 
military  map,  the  algorithms  used  in  the  map  reading  ICW  must  be  clearly 
explained.  Therefore,  the  algorithms  involved  were  limited  in  complexity  and 
applied  in  precisely  defined  situations.  This  made  the  problems  in  the  map  readi-ig 
ICW  comparatively  easy  to  understand  and  solve  given  the  quality  of  the  instrur.tion 
along  with  the  high  ability  level  of  the  subjects.  The  result  is  that  the  review  v  as 
merely  a  tool  to  aid  recall  instead  of  being  used  to  clarify  concepts.  This  is 
highlighted  by  89%  (31/35  S*s)  of  the  subjects  that  reported  on  the  post-treatment 
questionnaire  (Question  5)  that  they  primarily  used  the  review  to  look  up  answers 
for  the  fact  and  procedure  type  of  questions. 

As  alluded  to  above,  ability  level  may  have  also  been  a  factor.  Other  studies 
(Schaffer  &  Hannafin,  19c6;  Kinzie,  1987)  have  found  that  segregating  the  sample 
by  measured  learner  characteristics  had  an  effect  on  the  data  collected.  Since  the 
subjects  were  not  broken  down  by  ability  level  for  the  present  research  this  issue 
can  not  be  addressed.  It  is  quite  possible  that  by  expr  iding  the  present  research  to 
distinguish  subjects  that  represent  the  enlisted  population  in  the  Army,  which  have 
inconsistent  educational  backgrounds,  a  significant  effect  may  have  been  achieved. 
Therefore,  using  the  same  content  and  de^sign  factors  an  aptitude-treatment 
interaction  may  reveal  effects  that  were  not  found  in  the  present  study. 

The  last  limitation  falls  in  the  area  of  student  use  of  learner  control.  While  it 
was  noted  in  the  post-treatment  questionnaire  that  over  95%  of  the  subjects  were 
aware  that  the  option  to  review  the  course  content  there  is  no  indication  that  the 
subjects  knew  how  to  incorporate  the  review  option  into  their  study  habits. 
Therefore,  it  must  be  assumed  that  the  subjects  knew  how  to  make  the  best  use  of 
the  learner  control  afforded  them  in  this  study.  Clark  (1982)  reviewed  a  number  of 
studies  and  has  suggested  that  by  mistake  students  choose  methods  of  study  that 
inadvertently  result  in  less  learning.  In  other  words,  merely  providing  someone  a 
tool  to  do  a  job  is  no  guarantee  that  they  will  know  how  to  properly  use  the  tool. 
Clark's  review  taints  the  aforementioned  assumption  and  renders  it  tenuous  at  best. 

Recommendations 

Lack  of  significance  in  four  of  five  stated  hypotheses  could  lead  to  the 
conclusion  that  the  use  of  review  does  not  make  a  difference  in  a  student's  ability  to 
achieve.  Further,  providing  a  review  option  may  use  the  student's  time  to  no 
apparent  advantage.  Such  assumptions,  however,  should  be  tempered  by 
previously  mentioned  limitations  and  an  understanding  that  parallel  access  to 
information  is  a  recently  developed  feature  available  in  ICW.  As  learners  become 
more  familiar  with  learner  control  mechanisms,  then  the  effects  of  these 
mechanisms  (under  different  conditions)  may  become  apparent.  With  this  in  mind, 
the  following  recommendations  are  offered: 
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First,  ihc  length  of  time  a  student  must  devote  to  the  CAl,  in  one  sitting, 
should  be  limited  to  generally  accepted  practices  for  more  traditional  forms  of 
instruction  (i.e.,  a  10  minute  break  every  50  minutes),  so  that  fatigue  does  not  act 
as  an  confounding  variable.  This  can  easily  be  accommodated  by  the  computer 
software  so  that  subjects  can  continue  instruction  from  where  they  left  off. 
Considering  the  effects  computer  monitors  have  on  the  eyes,  and  considering  that 
text  presented  on  monitors  impain  reading  efficiency,  there  is  reason  to  study  how 
these  factors  affect  attention. 

Second,  the  presence  of  external  motivators  should  be  examined  as  an 
independent  variable  with  the  use  of  review  since  the  post-treatment  questionnaire 
indicated  that  subjects  would  be  more  inclined  to  use  the  review  if  external 
motivators  (i.e.,  minimum  score)  were  applied. 

Tliird,  further  research  in  this  area  should  avoid  the  use  of  multiple  choice 
questions  since  this  type  of  question  does  not  adequately  measure  a  students  ability 
to  recall  infonnation,  or  demonstrate  understanding  of  higher  order  rule  using.  If 
the  embedded  questions  are  designed  to  use  more  sophisticated  question  styles  then 
an  increase  in  programming  complexity  will  be  needed.  Again,  the  results  taken 
from  the  post-treatment  questionnaire  suggests  that  changing  this  part  of  the 
research  would  cause  an  increase  in  the  use  of  the  review.  The  unanswered 
question  is;  "Would  an  increase  in  the  use  of  the  review  cause  an  increase  in  ^^tudent 
achievement?" 

Fourth,  since  the  content  of  the  instruction  used  for  the  present  research  did 
not  seem  to  break  the  threshold  of  effectiveness  that  a  simple,  linear  ICW  design 
provided;  a  modification  should  be  made  to  the  research  design.  An  effort  should 
be  made  to  examine  the  effect  a  content  review  option  has  on  a  students  ability  to 
learn  content  that  either  has  complex  algorithms,  or  requires  the  application  of 
heuristic  problem  solving  techniques.  It  is  possible  that  more  complex  content 
would  tender  results  that  support  the  use  of  a  content  review  option. 

Fifth,  further  research  should  be  conducted  taking  into  account  learner 
characteristics.  This  proposed  research  could  organize  the  subjects  according  to 
reading  ability,  verbal  learners,  non-verbal  learners,  or  spatial  ability  to  name  a  few. 
In  light  of  the  evidence  from  previous  research  that  has  examined  learner 
characteristics  as  an  independent  variable  this  could  prove  to  be  a  promising 
research  tack  to  pursue  since  the  best  instructional  method  for  one  learner  may  not 
necessarily  be  the  best  instnictional  method  for  another. 

Sixth,  research  should  be  undertaken  that  examines  the  effectiveness  of 
training  subjects  how  to  use  learner  control.  As  ICW  becomes  more  sophisticated 
student  control  will  also  increase.  As  a  result  of  this  increased  control  it  may 
become  necessary  to  teach  students  how  to  use  the  ICW  to  their  ar^vantage. 
Additionally,  the  development  of  consistent  design  features  which  are  used  to 
access  and  navigate  through  information  may  greatly  enhance  the  effectiveness  of 
learner  cc  ^trol  options.  In  order  to  best  evaluate  the  effectiveness  of  these  concepts 
this  reseix  h  may  need  to  be  conducted  in  a  situation  where  learner  characteristics 
are  also  controlled. 

Last,  further  research  should  be  conducted  that  examines  the  idea  that,  for 
different  content  domains,  there  may  exist  different,  optimal  design  structures.  If 
true,  then  instructional  designers  would  gain  a  valuable  too!  to  help  make  decisions 
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that  affect  the  use  of  resources  needed  for  constructing  ICW.  The  value  of  this  is 
emphasized  by  instructional  designers,  in  industry  and  the  military,  propensity  to 
heavily  weight  efficiency  as  a  design  criterion.  TTie  present  research  being  an 
excellent  case  in  point.  Considering  that  the  LACRO  group  spent  significantly  less 
time  (M  =  74.02  min.)  processing  the  CA!  than  the  OACRO  group  (M  =  83.38 
min.)  with  no  discernable  difference  in  achievement  then,  firom  an  instructional 
designers  perspective,  it  makes  little  sense  to  expend  the  additional  resources 
required  in  designing,  installing,  and  evaluating  the  content  review  option. 
Therefore,  having  the  ability  to  prescribe  the  combination  of  design  factors  that 
effectively  communicate  a  certain  topic  to  the  student,  enhances  efficiency  in  the 
planning  and  development  phases  of  the  instructional  design. 

When  you  consider  that  the  goal  of  professional  trainers  is  to  expeditiously 
and  by  least  cost  produce  a  person  with  the  skills  necessary  to  be  able  to  function  in 
an  occupational  area  and  that  experience  is  still  considered  the  best  teacher  then 
there  is  a  strong  argument  for  using  modest  coursev/are  designs  for  certain  topics. 
Unpretentious  courseware  design  have  proven  effective  in  numerous  cases  and 
should  be  recognized  for  their  elegance  instead  of  being  chided  for  not  fulfilling  the 
expectations  of  those  that  do  not,  on  a  day-to-day  basis,  deal  with  the  complex 
issues  of  the  training  arena. 
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Figure  1 

Limited  Access  to  the  Content  Review  Option  (LACRO) 
(*  After  review  student  returns  to  point-of-acce:;s.) 
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Figure  2 

Open  Access  to  the  Content  Review  Option  (OACRO) 
(*  After  review  student  returns  to  point-of-access.) 
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Table  1 

Means.  Standard  Deviations,  and  Group  Sizes  Obtained  for  Total  Posttesi  Scores 


Group 

n 

Means 

S.D. 

LACRO 

35 

16.829 

3.044 

OACRO 

35 

15.914 

2.832 

Maximum  score  =  21. 
Table  2 


Summary  Table  of  the  Univariate  Analysis  of  Variance  Using  the  Posttest  Total 
Score  Data 


Source  of 

DF 

Sum  of 

Mean 

F 

Variation 

Squares 

Squares 

Ratio 

Group 

1 

14.629 

14.629 

1.693 

Within  Cells 

68 

587.714 

8.643 

ERIC 
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Table  3 


Posttest  Means.  Standard  Deviations,  and  Group  Sizes  Obtained  for  Fact. 
Procedure,  and  Problem  Solving  Achievement  Scores 


Question  Type 


Means 


S.D. 


Fact 

Procedure 
Prob.  Solving 


LACRO  (n  =  35) 

5.486 
5.971 
5.429 


1.147 
1.043 
1.632 


Fact 

Procedure 
Prob.  Solving 


OACRO  (n  =  35) 

5.171 
5.714 
5.229 


1.200 
1.126 
1.505 


Maximum  score  for  each  sub.scale  =  7. 
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Table  4 


1 


Suminarv  Table  of  the  Univariate  Analysis  of  Variance  Using  the  Posttest  Fact. 
Procedure,  and  Problem  Solving  Score  Data 


Source  of 
Variation 


Group 
Within  Cells 


DF 


1 

68 


Sum  of 
Squares 


Mean 
Squares 


Fact  Question 

1.729 
93.714 


1.729 
1.378 


F 

Ratio 


l.?54 


Group 
Within  Cells 


Procedure  Question 
1  1.157  1.157 

68  80.114  1.178 


.982 


Group 
Within  Cells 


Problem  Solving  Question 
1  2.800  2.800 

68  153.543  2.258 


1.240 
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Table  5 

Means.  Standard  Deviations,  and  Group  Sizes  Obtained  for  Acquisition  Rate 

Group  II  Means  S.D. 

LACRO  35  .243  .085 

OACRO  35  .206  .083 


Table  6 


Summary  Table  of  the  Analysis  of  Variance  Using  the  Acquisition  Rate  Data 


Source  of 

DF 

Sum  of 

Mean 

F 

Variation 

Squares 

Squiu-es 

Ratio 

Group 

1 

.025 

.025 

3.571 

Within  Cells 

68 

.483 

.007 
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Tabic  7 


Means  and  Standard  Deviations  for  LACRO  and  OACRO  Lesson  Times 


Group  Means  S.D.  » 

LACRO  (n  =  35) 


Lesson  1 

3.475 

2.053 

Lesson  2 

9.478 

4.208 

Lesson  3 

4.995 

2.199 

Lesson  4 

7.153 

2.442 

Lesson  5 

8.825 

2.977 

Lesson  6 

4.230 

2.689 

Lesson  7 

3.834 

2.918 

Lesson  8 

5.503 

4.455 

Lesson  Total 

47.491 

18.248 

OACRO  (n  =  35) 

Lesson  1 

4.276 

2.105 

Lesson  2 

10.939 

4.106 

Lesson  3 

5.890 

2.352 

Lesson  4 

7.543 

2.619 

Lesson  5 

9.756 

2.353 

Lesson  6 

5.171 

2.087 

Lesson  7 

5.003 

2.935 

Lesson  8 

6.380 

4.262 

Lesson  Total 

54.958 

16.169 
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Table  8 

Summary  Table  of  the  Univariate  Analysis  of  Variance  Using  the  Lesson  Time  Data 

Source  of  DF           Sum  of  Mean  P 

Variation  Squares  Squares  Ratio 

Lesson  I 

Group  1             11.246  11.246  2.601 

Within  Cells  68           293.972  4.323 

Lesson  2 

Group  1             37.369  37.369  2.163 

Within  Cells  68          1175.056  17.280 

Lesson  3 

Group  1              14.033  14.033  2.708 

Within  Cells  68           352.374  5.182 

Lesson  4 

Group  1               2.673  2.673  .417 

Within  Cells  68           435.965  6.411 

(continued  on  next  page) 
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Table  8  (coiit, ) 
Lesson  5 

Group  1             15.162  15.162  2.106 

Within  Cells  68           489.671  7.201 

Lesson  6 

Group  1              15.490  15.490  2.674 

Within  Cells  68            393.955  5.793 

Lesson  7 

Group  1             23.904  23.904  2.792 

Within  Cells  68            582.265  8.563 

Lesson  8 

Group  1             13.478  13.478  .709 

Within  Cells  68          1292.599  19.009 

Ixsson  Total 

Group  1            975.724  975.724  3.283 

Within  Cells  68         20209.796  297.203 


ERIC 


020 
32 


20 

Table  9 


Means.  Standard  Deviarions.  and  Group  Sizes  Obtained  for  Overall  Time  Using 
CM  (Course  Time^ 


Group 

n 

Means 

S.D. 

LACRO 

35 

74.018 

18.868 

OACRO 

35 

83.375 

18.850 

Table  10 


Summary  Table  of  the  Analysis  of  Variance  Using  the  Course  Time  Data 


Source  of 

DF 

Sum  of 

Mean 

F 

Variation 

Squiu-es 

Squares 

Raiio 

Group 

1 

1532.429 

1532.429 

4.309  * 

Within  Cells 

*         _  A/IT 

68 

24185.214 

355.665 

*  p  =  .042 
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Table  1 1 


Summary  Table  of  the  Analysis  of  Covariancp,  of  Total  Course  Time  and  Review 
Time  Data  as  the  Coyariate 


Source  of  DF  Sum  of  Mean  F 

Variation  Squares  Squares  Ratio 

Group  1  275.748  275.748  >  .770 
Review 

Time  1  183.041  183.041  .51  1 

Within  Cells  67  24002.172  358.241 
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A  Research  Approach  to  the  Identification, 
Clarification,  and  Definition  of  Visual  Literacy 
and  Related  Concepts 


Abstract 

This  paper  presents  a  report  of  a  multi-phase  research  project  which  was  conducted 
to  provide  a  comprehensive  description  of  visual  literacy,  its  constructs  and 
components.  The  Delphi  technique  was  employed  to  elicit  information  and  opinions 
from  a  panel  of  visual  literacy  experts,  A  set  of  statements  based  upon  the  initial 
input  of  that  group  was  further  refined  by  the  group  through  successive  Delphi 
iterations.  While  the  Delphi  process  was  central  to  this  research,  basic  Delphi 
techniques  were  expanded  to  include  an  extensive  instrumentation  phase  and  a 
validation  of  the  results  of  the  Delphi  study  through  a  readministration  of  the 
instrument  with  a  different  group  of  subjects.  A  brief  disjussion  of  the  background 
of  visual  literacy  and  a  description  of  the  Delphi  methods  used  are  included, 
followed  by  a  report  of  the  procedures  employed  in  this  project  and  the  results  of  the 
study. 
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A  Research  Approach  to  the  Identification, 
Clailfication,  and  Definition  of  Visual  Literacy 
and  Related  Concepts 


Introduction 

Visual  literacy  is  a  familiar  term  to  the  majority  of  individuals  in  the  fields  of  education 
and  educational  technology.  However,  the  term  may  mean  far  different  things  to  different 
individuals.  For  more  than  two  decades  the  charm  of  visual  literacy  as  a  general  notion  has 
exceeded  its  popularity  as  a  defined  (ill  defined?)  entity,  ^s  visual  literacy  a  set  of  skills?  Is  it 
a  field  of  study,  an  academic  discipline?  Is  it  a  metaphor,  a  misnomer,  or  just  a  mystery? 
These  and  many  other  questions  regarding  visual  literacy  have  lacked  definitive  answers. 
This  paper  presents  a  report  of  a  multi-phase  research  project  which  was  undertaken  to 
address  some  of  the  questions  surrounding  visual  literacy. 

Origins  of  Visual  Literacy 

In  1%9  the  first  visual  literacy  conference  was  held  at  Rochester,  New  York.  John 
Debes  was  the  driving  force  behind  that  meeting  and  his  seminal  paper.  The  Loom  of  Visual 
Literacy"  (Debes,  1970),  set  the  ^/isual  literacy  movement  in  motion.  The  idea  of  visuals  as  a 
language  was  recorded  in  an  L^^^^^nan  Kodak  publication  in  197  L  Visual  thinking  was 
advanced  as  a  related  concept  the  next  year  (Amheira,  1972).  And  the  following  year  Denis 
Dondis'  classic  text,  A  Primer  of  Visual  Literacy,  appeared  (Dondis,  1973).  By  1976  the  term 
visual  literacy  (VL)  had  become  a  buzzword.  In  August  of  1976  the  theme  of  the  twenty- 
second  Lake  Okoboji  Educational  Media  Leadership  Conference  was  "Visual  Literacy  -  The 
Last  Word"  (Cureton  &  Cochran,  1976).  But  it  wasn't  the  last  word.  Although  the  Okoboji 
participants  stated  that  "Before  research  can  begin,  a  common  definition  of  the  term  Visual 
literacy'  should  be  developed",  (p.  99),  no  widely  accepted  (thus  common)  defuiition  resulted 
from  that  conference. 

Attempts  at  Definition 

Debes  himself  initiated  the  ongoing  definitional  troubles.  He  wrote  that  "When  I  say 
Visual  literacy,  what  I  have  in  mind  is  a  great  dim  shape,  the  outlines  of  which  and  the 
importance  of  which  are  not  yet  clear"  (1970,  p.  1).  Debes  agreed  to  write  the  first  definition 
of  VL  with  the  caveat  that  the  definition  was  tentative  and  "sure  to  be  changed"  (1970,  p.  14). 
Dondis  may  have  reinforced  the  general  reluctance  to  create  a  sharply  focused  common 
definition  when  she  advised,  "One  of  the  major  pitfalls  in  developing  an  approach  to  visual 
literacy  is  trying  to  overdefine  it  (1972,  p  9)." 

Since  those  early  days,  the  definitions  of  VL  have  varied  widely  Some  of  the  more 
recent  definitions  include  only  a  narrow  range  of  elements  while  others  contain  many.  Some 
writers  and  practitioners  have  addressed  only  understandmg  visuals,  others  have  included 
understanding  and  creating  visuals,  while  still  others  have  included  a  much  broader  range  of 
abilities.  Hortin,  for  instance,  stated  that  VL  mcludes  the  abilities  required  to  "learn, 
understand,  think,  communicate  and  store  information  in  one's  mind  visually"  (1980,  p.  21). 
In  a  definition  deliberately  structured  to  incorporate  the  essentials  of  visual  thinking,  visual 
learning,  and  visual  language,  Hortin  and  Braden  offered  this  definition:  "Visual  literacy  is 
the  ability  to  understand  and  use  images,  including  the  ability  to  think,  learn,  and  express 
oneself  in  terms  ot  images"  (1982,  p»  169).  As  Baca  and  Braden  noted,  "even  that  definition 
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fails  to  directly  address  design,  creativity,  and  aesthetics  as  they  apply  to  visualization."  (1990, 
p.  100). 


While  the  term  visual  literacy  has  endured,  after  those  first  few  years  the  concept  has 
gained  few  new  advocates.  There  has  been  no  disputing  the  fact  that  visuals  can  be  used  in 
communication.  There  has  been  no  challenging  of  the  basic  premise  that  principles  related 
to  using  and  understanding  visuals  can  be  taught  and  learned.  The  inherent  wholesomeness 
of  VL  remains,  but  not  without  challenge. 


Criticisms 

Over  the  years  visual  literacists  have  lauded  the  precepts  of  visual  literacy,  while  others 
increasingly  have  asserted  that  VL  is  not  a  viable  concept.  The  detractors  have  made  an  issue 
of  rejecting  VL  entirely.  Suhor  and  Little  suggested  that  VL  is  "...not  a  coherent  area  of 
study  but,  at  best,  an  ingenious  orchestration  of  ideas"  (1988,  p.  ^1^)-  Cassidy  and  Knowlton 
labelled  visual  literacy  a  "failed  metaphor"  and  offered  the  proposition  that  "Visual  Literacy 
(VL)--as  a  metaphor,  as  a  concept,  and  as  a  movement-is  a  paradox.  ...in  the  sense  of  [being] 
inherently  contradictory".  They  asked  "What  is  VL,  or,  by  the  criterion  of  a  profitable  science 
arid  technology,  what  might  it  become  if  it  is  to  serve  as  an  abundant  generator  of  questions, 
hence  a  guide  to  fruitful  research?"  (1983,  p.  67). 

The  aspects  of  VL  which  are  most  frequently  criticized  include  its  theoretical  foundations 
and  the  research  which  has  been  conducted  in  the  field.  Hocking  (1978)  addressed  the 
paucity  of  experimental  research  in  the  field  thus: 

Much  of  the  literature  available  fell  within  the  reahn  of  hypothesizing  and 
undocumented  personal  opinion.  ...Some  research  has  been  background 
research  concerning  the  rationale  behind  Visual  Literacy;  and  much  of  the 
field  research  has  been  without  the  tight  control  of  experimental  design. 
...the  tremendous  amount  of  information  published  on  a  topic  that  crosses  so 
many  fields  and  disciplines  prevents  any  hope  of  exhaustiveness...  (pp.  10-1 1). 

Clearly  these  and  other  criticisms  relate  to  the  expansiveness  of  VL  and  the  failure  of 
the  field  to  clearly  identify  its  component^.  Description  and  definition  are  at  the  root  of 
many  of  the  objections  to  VL.  A  comprehensive  description  of  the  components  of  visual 
literacy  and  cf  the  ideas  related  to  it  might  provide  some  structure  for  research  that  would 
validate  the  field.  Cassidy  and  Knowlton  called  for  a  dcvscription  of  visual  literacy  which  could 
provide  direction  for  future  research.  They  stated  that  "...if  from  the  VL  arena  someone 
would  step  forward  and  offer  conceptual  and  operational  defimtioiK  of  VL  (no  matter  how 
incomplete  and  tentative),  honest-to-goodness  research  work  could  begin"  (1983,  p  84). 
Spitzer  and  McNemy  called  for  an  operational  definition  of  visual  literacy,  claiming  that 
without  one  "...the  nature  of  the  construct  is  speculative,  ambiguous,  and  too  prone  to 
misinterpretation*  f 1975,  p.  3 1).  Hocking  stated  that  'The  enumeration  of  the  skills  of  Visual 
Literacy  [would]  help  define  the  discipline"  (p.  13). 


Proliferation  of ''literacies'' 

First  there  was  literacy.  It  had  to  do  with  the  ability  to  read  and  write  words.  Then  the 
concept  was  expanded  to  include  aspects  of  being  well  read.  When  the  term  "visual  literacy" 
was  coined  the  conceptual  door  was  thrown  open.  Other  adjectives  have  been  linked  since  to 
literacy,  the  most  ubiquitous  being  "computer."  Recently  the  term  media  literacy  has  been 
introduced    and,  as  defined  by  Tyner  (1989),  that  term  would  subsume  VL. 
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As  recently  as  the  autumn  of  1989  a  call  for  papers  for  a  London  symposium  contained 
the  following  opening  paragraph: 


The  term  "literacy"  is  invoked  by  diverse  groupings  of  academics, 
educationalists  and  media  producers  who  want  to  s  ^^^tablished  concerns 
with  print  media  extended  to  cover  the  fields  of  tele\  Qlm,  radio,  video, 
audio,  photography,  graphic  design  and  computer  iol  vare.  But  do  we  all 
mean  the  same  things  by  "literacy"?  What  is  entailed,  conceptually  and 
politically,  in  extending  its  usage  to  all  r'^dia  forms?  (IVLA,  n.d.) 

Proliferation  of  literacy  terms  only  compounds  the  identity  problems  for  VL.  Yet  the 
phenomenon  has  its  lighter  side.  One  of  the  participants  in  this  research  used  the  term  sexual 
literacy,  but  no  reference  to  it  has  been  noted  in  the  literature.  (We  can  only  hope  that  the 
latter  term  does  not  survive.  How  would  we  know  what  to  classify  as  reading  and  what  as 
writing?) 

Many  of  the  scholars  who  have  addressed  the  theoretical  foundations  of  VL  have 
asserted  that  the  basis  for  the  concept  of  visual  literacy  has  oeen  derived  by  borrowing  from  a 
diversity  of  fields,  including  education,  philosophy,  perception,  art  theory,  linguistics,  and 
human  developmental  theory  (Debes,  1970;  Hocking,  1978;  Hortin,  1980).  In  the  process, 
however,  VL  has  not  [in  the  opinion  of  the  present  authors]  created  a  substantial  theory  base 
or  research  identity  of  its  own  that  commands  academic  respect. 

Research  Goals 

The  goals  of  this  study  were  to  identify  the  constructs  of  visual  literacy,  to  determine  the 
amount  of  agreement  or  disagreement  which  exists  concerning  the  identified  constructs,  and 
to  provide  a  measure  of  relative  importance  among  the  "dentified  constructs.  The  following 
research  questions  were  developed  to  enable  these  goals  to  be  met. 

1.  What  are  the  elements  of  visual  literacy? 

2.  How  much  agreement  exists  concerning  the  identification  of  the  constructs 
which  are  elements  of  visual  literacy? 

3.  Given  the  constructs  identified  as  elements  of  visual  literacy,  how 
important  is  each  construct  considered  to  be  by  persons  active  in  the  visual 
literacy  'leld? 

The  Delphi  Method 

The  Delphi  technique  is  a  research  method  which  is  used  to  elicit  the  input  of  a  group  of 
individuals  and  to  pool  their  collective  knowledge  in  o'-der  to  formulate  a  group  judgment  or 
statement  of  opinion.  The  process  is  designed  to  maximize  human  intellect  while  attempting 
to  minimize  interpersonal  dpamics.  A  major  anticipated  effect  of  the  procedure  is  to 
generate  a  better  estimate  or  a  more  sharply-focused  opinion  than  can  be  expected  from  a 
single  individual  or  from  a  group  dominated  by  and  individual. 

Orii^inal  Purpose 

llie  Delphi  technique  was  originally  used  for  forecasting  future  developments.  One 
notal  ^e  example  of  such  an  applicatici  was  the  use  of  the  method  by  the  United  States 
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military  for  long-term  threat  assessment  (Dalkey,  1969).  When  the  Rand  Corporation  first 
conducted  studies  for  the  Department  of  Defense,  they  also  conducted  extensive  tests  to 
validate  the  methodology  and  reported  these  tests  in  a  series  of  reports  (Dalkey,  1969; 
Brown,  Cochran  &  Dalkey,  1969;  Dalkey,  Brown  &  Cochrr.n,  1969;  Dalkey,  Brown  & 
Cochran,  1970). 

The  validation  studies  performed  by  the  Rand  Corporation  utilized  "almanac  type" 
(factual)  questions  with  identifiable  "correct"  (quantitative)  answers.  The  studies  showed 
that,  with  reiteration,  the  group  estimate  (identified  as  the  median  of  the  responses)  became 
more  accurate.  Further,  the  diversity  of  answers  narrowed  toward  a  greater  degree  of 
consensus  or  agreement  (Dalkey,  1969,  p.  20). 

Other  Uses 

The  Delphi  technique  is  now  commonly  applied  in  situations  in  which  there  exists 
knowledgeable  opinion  rather  than  questions  about  hard  information  for  which  there  are 
specific  correct  answers.  Recently,  the  Delphi  technique  has  been  applied  in  fields  such  as 
management,  education,  and  medicine.  It  is  used  as  a  decision  making  process  in  formulating 
policy  statements,  in  forecasting  future  technological  developments  of  technologies  and  their 
applications,  in  risk  assessment,  and  in  curriculum  development.  Not  only  is  the  technique 
being  used  in  a  growing  number  of  settings,  wherever  used,  the  Delphi  process  tends  to 
generate  statements  which  are  better  accepted  than  those  derived  through  direct  forms  of 
interaction  (Dalkey,  1969,  p.  17). 

The  Delphi  technique  is  particularly  useful  when  an  attempt  is  being  made  to  achieve 
one  of  the  following  purposes: 

...(a)  determining  or  developing  a  range  of  possible  alternatives,  (b)  exploring 
or  exposing  underlying  assumptions  or  information  leading  to  various 
judgments,  (c)  seeking  out  information  that  may  generate  a  consensus  on  the 
part  of  the  group  involved,  or  (d)  correlating  informed  judgments  on  topics 
spanning  several  disciplines  (Harrison,  1987,  p.  278). 

Thus,  the  Delphi  technique  appeared  to  have  promise  for  resolving  the  issue  of  visual 
literacy.  The  techniques  were  assumed  to  be  an  appropriate  method  for  correlating  informed 
judgments  and  identifying  (and  perhaps  generating)  consensus  among  scholars  who  stu  V 
visual  literacy. 


Procedure  for  this  Study 

In  the  research  reported  here,  the  Delphi  technique  was  the  primary  method  employed 
to  de/elop  a  comprehensive  description  of  visual  literacy.  However,  this  research  project  was 
expanded  beyond  the  Delphi  study  to  include  two  phases.  Phase  one  began  vvdth  the 
development  of  an  instrument  which  could  be  used  to  poll  the  opinions  of  a  group  of  experts 
who  participated  in  a  Delphi  study.  The  reactive  phase  of  the  study  followed,  in  which  the 
participants  rated  the  statements  on  the  instrument.  Phase  two  validated  the  results  of  the 
Delphi  study  by  asking  a  different  population  to  rate  the  items  on  the  same  instrument. 


Phase  I:  The  Delphi  Study 

The  procedures  for  selecting  participants  and  developing  the  instrument  were  somewhat 
more  elaborate  than  those  described  by  Isaac  &  Michael  (1981,  p.  114415).  Their  five  step 
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procedure  for  conducting  a  Delphi  study  was  followed,  except  the  opinion -gathering  step  was 
terminated  at  two  rounds  when  early  consensus  was  reached. 


Population  The  Delphi  technique  is  a  method  for  pooling  the  opinions  of  a  group  of 
experts.  Therefore,  the  selection  of  subjects  who  would  participate  in  this  study  involved  the 
identification  of  a  group  of  persons  who  could  be  considered  experts  on  the  topic  of  visual 
literacy.  This  selection  was  accomplished  in  the  following  manner: 

1.  A  list  of  names  was  compiled  which  included: 

a.  all  the  people  who  had  published  one  or  more  articles  in 
The  Journal  of  Visual-Verbal  Languaging  (JWL)  in  the 
previous  five  years, 

b.  the  people  who  had  published  in  the  readings  from 
International  Visual  Literacy  Associction  (IVLA)  annual 
conferences  between  1984  and  1988,  and 

c.  the  participants  at  the  IVLA  Symposium  on  Verbo  Visual 
Literacy  which  was  held  in  Stockholm,  Sweden,  in  1987. 

2.  This  list,  which  included  two  hundred  seventeen  names,  was  sent  to  the 
eight  members  of  the  1988  Executive  Board  of  the  International  Visual 
Literacy  Association.  Each  member  was  asked  to  select  from  the  list  and  to 
add  other  names  in  order  to  identify  thirty  persons  whom  they  felt  were 
knowledgeable  about  visual  literacy  or  had  made  significant  contributions  to 
the  field  and  could  be  considered  "experts".  A  total  of  eighty-eight  people 
were  identified  as  having  a  degree  of  expertise  adequate  to  be  asked  to 
participate  in  the  Delphi  study. 

An  invitation  to  participate  in  the  Delphi  study  was  sent  to  each  of  the  eighty-eight 
people  identified  in  this  manner.  The  letter  of  invitation  included  a  brief  discussion  of  the 
Delphi  technique  and  emphasized  that  a  Delphi  study  is  a  long  process  for  both  the 
researcher  and  the  participants.  The  recipients  of  this  letter  were  asked  whether  or  not  they 
were  able  and  willing  to  commit  the  time  which  would  be  required  to  complete  the  study.  Of 
the  eighty-eight  invitations  which  were  sent,  seventy-three  responses  were  received;  fifty-nine 
of  those  responses  were  from  people  who  agreed  to  participate  in  the  study. 

Instrumentation.  A  review  of  the  literature  was  conducted  to  identify  questions, 
concerns,  and  ideas  which  had  been  addressed  as  aspects  of  visual  literacy.  Based  upon  this 
literature  review,  twenty-six  broad  questions  were  formulated.  These  questions  were  to  be 
used  as  stimuli  for  the  respondents'  statements  of  opinion  regarding  visual  literacy  constructs. 
A  final  question  was  added  which  asked  the  participants  to  identify  areas  of  concern  which 
were  not  addressed  by  the  stimulus  questions.  This  final  question  and  the  unstructured 
format  for  the  responses  were  utilized  in  order  to  take  this  study  beyond  the  limits  of  the 
existing  literature  and  to  benefit  fully  fi:om  the  expert  knowledge  and  opinions  of  the  Delphi 
participants. 

The  participants  were  asked  to  respond  to  the  questions  with  written  answers.  This 
diverges  somewhat  from  the  procedure  usually  employed  in  a  Delphi  study.  It  is  more 
common  for  the  researcher  to  formulate  a  tentative  list  of  statements,  thus  making  analysis  of 
the  responses  much  more  manageable.  In  this  case,  providing  such  a  list  might  have  limited 
the  ideas  and  constructs  addressed  by  the  participants  and  was,  therefore,  judged  not  to  be 
congruent  with  the  goal  of  providing  a  comprehensive  description  of  visual  literacy. 
However,  since  the  responses  were  to  be  written  by  each  participant,  asking  busy 
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professionals  to  respond  to  twenty-seven  questions  would  have  made  unreasonable  demands 
on  their  time.  Tnerefore,  the  questions  were  divided  and  six  sub-forms  were  created,  each  of 
which  contained  seven  questions.  Of  those  seven  questions,  five  were  asked  of  only  some  of 
the  participants,  while  the  two  broadest  questions  were  on  each  sub-form.  Tliese  two 
questions,  asked  of  every  participant,  were: 

Provide  a  definition  of  visual  literacy  which  is  acceptable  to  you.  It 
may  be  your  own,  someone  else's,  or  one  which  you  have  modified.  Some 

elements  you  may  wish  to  include  are:  abilities  to  with  (elements)  for 

(purposes)  using  (tools)  

What  other  concerns,  opinions,  quesrions  etc.  regarding  the  nature 
and/or  the  limits  of  visual  literacy  would  you  like  to  see  addressed  in  this 
study?  You  might  consider  things  I  have  left  out  in  regard  to  a  definition  of 
the  term,  visual  literacy  as  a  field,  visual  language,  the  practice  of  visual 
literacy,  educational  implications,  and  visual  literacy  research. 

Each  participant  was  sent  the  i»*aster  list  of  all  twenty-seven  questions  and  a  shorter  form 
consisting  of  seven  questions.  Participants  were  asked  to  answer  all  the  questions  on  their 
short  form  and  as  many  from  the  master  list  as  their  time  and  interest  allowed.  The  final 
question  on  each  short  form  asked  every  participant  to  examine  the  master  list  in  order  to 
identify  aspects  of  visual  literacy  which  were  not  already  being  adequately  addressed.  In 
addition,  an  extensive  list  of  terms  which  might  serve  as  stimuli  was  included.  This  list 
included  terms  which  could  be  identified  ia  the  literature  as  related  to  visual  literacy.  Round 
One  of  the  Delphi  study  thus  consisted  of  a  master  list  of  twenty-seven  questions,  one  of  six 
short  forms  which  each  included  seven  questions,  a  list  of  terms,  and  a  cover  letter.  This  was 
sent  to  the  fifty-nine  participants  in  May,  1989. 

Fifty-four  responses  to  the  questionnaire  used  in  Round  One  were  received.  Each 
statement  (many  of  which  were  quite  lengthy)  was  broken  into  individual  concepts.  More 
than  1,600  concepts  were  identified  in  the  responses  to  Round  One.  Because  of  the  large 
number  of  items  which  needed  to  be  included,  the  for.n  was  organized  by  grouping  portions 
of  statements.  An  effort  was  made  to  preserve  as  much  of  the  original  language  of  the 
participants  responses  as  possible.  This  form  included  broad  areas  (constructs)  and  as  many 
supporting  details  as  could  be  identified  and  organized  into  the  fi-amework  provided  by  the 
constructs.  When  completed,  the  form  contained  four  hundred  ninety-one  items.  That 
number  was  far  too  many  considering  that  the  participants  were  volunteers  who  might  be 
overpowered  by  the  task  of  evaluating  so  many  statements. 

The  concepts  were  collapsed  once  more,  losing  some  but  not  all  of  the  participants* 
original  phrasing.  The  revised  form  was  structured  as  a  questionnaire  with  186  items  -  still 
more  items  than  desired,  but  much  better  than  the  trial  form.  Items  were  listed  as  statements, 
each  to  be  rated  for  agreement/disagreement  on  a  four-point  Likert-type  scale.  This  final 
form  was  the  opinion  gathering  instrument  used  in  the  next  two  Delphi  rounds. 


Data  Collection 

On  the  two  iterations  of  the  Delphi  study  which  utilized  this  instrument,  the  participants 
were  asked  to  rate  each  statement  (or  portion  of  a  statement)  on  a  scale  fi:om  1  to  4, 
indicating  the  extent  to  which  they  agreed  that  the  statement  identified  or  elaborated  upon 
an  element  of  visual  literacy.  The  respondents  were  asked  to  use  the  following  values  for  the 
scaled  responses.  A  response  of  "4"  indicated  strong  agreement  ("agree  strongly"),  "3" 
indicated  mild  agreement  ("agree  somewhat"),  "2"  indicated  mild  disagreement  ("disagree 
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somewhat"),  and  indicated  strong  disagreement  ("disagree  strongly").  In  Round  Two, 
participants  were  encouraged  to  make  written  comments  or  to  suggest  re-phrasing  of 
statements. 


Data  Analysis  Methods 

The  distinctive  characteristics  of  a  Delphi  study  are  1)  no  face  to  face  interaction 
between  participants  during  the  study,  and  2)  reiteration  with  statistical  feedbac'ic  The 
statistical  feedback  consists  of  a  summary  of  the  group's  previous  responses,  provided  by 
including  the  extremes  of  the  interquartile  range  (Qj  and  Q3)  and,  for  each  individual,  his  or 
her  previous  response  to  each  item.  Including  the  first  and  third  quartiles  on  the  next  round 
of  the  Delphi  study  gives  the  participants  information  on  the  range  of  answers  selected  by  the 
central  fifty  percent  of  the  group  (from  Qj  to  Q3).  This  information  was  added  to  each  item 
for  Round  TTiree. 

In  addition  to  feedback,  the  Delphi  procedure  requires  data  for  the  determination  of 
level  of  agreement  and  degree  of  agreement  for  every  item  considered.  Consequently,  a  set  of 
simple  statistics  was  generated  from  the  Round  Two  responses: 

Statistic  Purpose 

Ql      upper  limit  of  the  first  quartile  feedback 

Q3      upper  limit  of  the  third  quartile  feedback 

IQR    the  interquartile  range  degree  of 

consensus 

M       mean  rating  level  of 

agreement 

For  Round  Three  the  IQR  and  M  were  computed.  Qj  and  Q3  were  available  but  were 
not  utilized  since  the  Delphi  procedure  was  terminated  prior  to  a  fourth  round. 

On  Round  Three,  the  Delphi  participants  were  asked  to  re-rate  each  statement,  this 
time  considering  the  group's  response  and  their  own  response  on  the  previous  round.  They 
were  also  given  the  opportunity  to  make  comments  about  the  items  for  which  their  rating  fell 
outside  the  interquartile  range.  Forty-three  responses  to  Round  Three  were  completed  and 
returned.  The  interquartile  range  and  the  mean  for  each  item  was  computed  from  these 
responses. 

A  /-test  was  performed  on  the  summed  means  of  all  the  items  on  Round  Two  and  the 
summed  means  of  all  the  items  on  Round  ITiree  to  determine  whether  a  statistically 
significant  change  in  the  ratings  had  occurred.  There  was  found  to  be  no  significant 
difference  (f=1.81,  df=91,  p.=.074)  between  Round  Two  (M=632.58,  n=50)  and  Round 
Three  (M =657.33,  n=43);  therefore,  the  Delphi  study  was  concluded.  (Had  the  difference 
been  significant,  another  round  would  have  been  administered.) 

Phase  II:  Validation 

Phase  II  of  this  research  project  utilized  a  different  population  and  involved  re-ratibg 
the  visual  literacy  concepts  identified  by  the  Delphi  population.  There  were  two  goals  for 
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this  phase  of  the  project.  The  first  was  to  validate  the  Delphi  study  by  asking  another  group 
to  rate  the  items  on  the  instrument.  The  second  goal  was  to  provide  a  measure  of  importance 
of  the  items  which  were  ace  pted  as  constructs  of  visual  literacy  during  the  Delphi  study. 


PopulatioTU  Since  a  major  goal  of  Phase  11  was  to  validate  the  results  of  the  Delphi  study, 
it  was  important  that  the  subjects  involved  in  this  phase  be  both  concerned  with  and  informed 
about  visual  literacy.  Therefore,  the  members  of  IVLA  (as  of  October  1989)  who  had  not 
participated  in  the  Delphi  study  were  used  as  the  population.  Questionnaires  were  sent  to 
one  hundred  eighty  people. 

Instrumentation  and  data  collection.  The  list  of  statements  which  >/as  used  in  the  Delphi 
study  was  used  as  the  instrument  for  this  phase  of  the  study.  Beca' ise  one  of  the  goals  of 
Phase  n  was  to  validate  the  Delphi  study,  the  items  which  had  been  rejected  by  the  Delphi 
participants  (those  with  a  mean  of  less  than  3.0  on  Round  Three)  were  included  in  this  phase. 
The  statements  were  still  accompanied  by  a  four  point  Likert-type  scale,  but  this  time  the 
respondents  were  asked  to  rate  each  item  as  to  how  important  they  felt  each  item  to  be. 

Data  analysis  methods.  The  responses  to  the  instrument  during  the  validation  phase 
were  compared  with  the  final  responses  to  the  Delphi  study  to  detenr^ne  there  was  a 
significant  difference  in  the  ratings  of  ihe  items.  A  f-test  was  performed  ol  .he  summed 
means  of  the  responses  to  all  the  items  to  determine  whether  there  was  a  significant 
difference  in  the  responses  to  the  instrument  as  a  whole.  In  addition,  a  r-test  was  performed 
on  each  individual  item,  comparing  the  mean  response  of  the  Delphi  participants  with  the 
mean  response  from  the  validation  group. 

An  F-rat^o  was  computed  to  determine  if  the  variance  in  the  responses  to  the  instrument 
during  the  validation  phase  was  significantly  different  from  the  variance  in  the  responses  to 
the  instrument  on  both  iterations  of  the  Delphi  study. 

Results 

Information  Resulting  from  the  Delphi  Study 

The  results  of  the  Delphi  study  provide  two  types  of  information  for  each  item:  1)  the 
degree  of  consensus,  and  2)  the  level  of  acceptance.  These  results  are  based  on  the  statistical 
analysis  performed  on  the  final  round.  The  interquartile  range  on  the  final  round  was  used  as 
the  measure  of  the  degree  of  consensus  concerning  whether  each  item  identifies  a  construct 
of  visual  literacy.  The  items  may  then  either  be  ranked  by  degree  of  consensus  or  arranged 
into  groups  by  degree  of  consensus.  The  level  of  acceptance  was  used  to  determine  the 
inclusion  or  exclusion  of  an  item  in  the  final  list  of  constructs  which  define  the  field  of  visual 
literacy. 

The  measure  of  the  degree  of  consensus  on  an  item  is  the  interquartile  range.  The 
interquartile  range  is  figured  by  subtracting  Qi  from  Q3,  hus  identifying  the  range  of 
responses  among  the  central  50%  of  respondents.  The  smaller  the  interquartile  range,  the 
greater  the  degree  of  consensus.  The  larger  the  interquartile  range,  the  less  the  degree  of 
consensus. 

The  measure  of  the  level  of  acceptance  for  each  item  is  the  measurement  of  group  rating 
of  an  item;  either  the  median  or  ^he  mean  for  each  item  may  be  used.  A  level  at  which  each 
item  is  accepted  as  a  construct  of  visual  literacy  was  established  for  this  study.  Items  meeting 
or  exceeding  that  level  were  accepted  as  a  construct  of  visual  literacy.  Items  falling  below 
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that  level  were  rejected  as  constructs  of  visual  literacy.  In  this  study,  to  be  accepted  an  item 
must  have  achieved  a  mean  of  at  least  2.98  to  be  accepted  (an  arbitrary  criterion  derived  by 
calculation  of  the  .05  alpha  level  of  all  means  of  the  final  round).  Any  item  with  a  lower 
mean  was  considered  to  have  been  rejected  by  the  Delphi  panel. 

The  level  of  acceptance  and  the  degree  of  consensus  may  be  independent  of  one 
another.  It  is  possible,  for  instance,  for  there  to  be  an  extremely  high  degree  of  consensus 
(interquartile  range  of  .5)  regarding  an  item  which  is  rated  low  enough  to  be  rejected 
(M=2.97  or  lower).  However,  the  correlation  between  the  level  of  acceptance  and  the 
degree  of  consensus  for  this  study  was  found  to  be  quite  high  (r=.908). 


Statistical  Analysis  of  the  Visual  Literacy  Delphi  Study 

Changes  in  Degree  of  Consensus.  In  order  to  determine  whether  or  not  there  had  been  a 
significant  convergence  toward  consensus  as  a  result  of  the  Delphi  process,  the  variance  on 
Round  Two  was  compared  with  the  variance  on  Round  Three.  A  decrease  in  variance  on 
Round  Three  would  indicate  a  convergence  toward  consensus;  such  an  observation  could 
then  be  tested  for  statistical  significance  by  computing  an  f -ratio. 

An  F-ratio  was  computed  to  determine  if  the  variance  in  the  responses  to  Round  Two 
was  significantly  different  from  the  variance  in  the  responses  to  Round  Three  of  the  Delphi 
study.  The  difference  in  the  variance  for  the  instrument  as  a  whole  (all  186  items)  was  found 
to  be  statistically  significant  (f =2.33,  df =49/42,  p=.003)  between  Round  Two  (var =5868.94, 
n=50)  and  Round  Three  (var=2521.04,  n=43).  The  decrease  in  the  variance  (computed  by 
squaring  the  standard  deviation)  shows  a  significant  move  toward  consensus. 


Non-Statistical  Results  of  the  Delphi  Study 

While  statistical  feedback  on  each  reiteration  of  a  Delphi  study  is  a  vital  component  of 
the  methodology,  the  results  of  such  a  study  are  not  necessarily  reported  statistically. 
Frequently  the  statements  are  grouped  and  identiCd  as  those  items  upon  which  there  exists 
high,  medium,  and  low  degrees  of  consensus.  An  alternative  is  to  rank  the  statements  in 
order  of  the  degree  of  consensus. 

For  many  researchers,  the  most  exciting  aspect  of  a  study  is  the  possibility  of  contributing 
to  the  resolution  of  the  question  which  stimulated  the  study.  The  opportunity  to  addrus  the 
questions  "What  does  this  mean?"  and  "So  what?"  is  usually  confined  to  the  discussion  in  ihe 
section  of  the  report  which  presents  conclusions  drawn  from  the  study.  However,  it  is 
important  to  keep  in  mind  that  the  goal  of  a  Delphi  study  is  to  create  a  statement  based  upon 
group  opinion  and  that  such  a  statement  is  the  true  "result"  of  a  Delphi  study. 

Arbitrarily,  the  authors  have  chosen  to  limit  discussion  of  specific  VL  constructs  to 
the  "top  ten  percent."  The  nineteen  statements  regarding  visual  literacy  elements,  concepts, 
or  constructs  for  which  there  was  the  highest  level  of  agreement  and  the  nineteen  statements 
about  which  there  was  the  highest  degree  of  consensus  ar'^  correlated  1:1.  Not  only  are  the 
statements  the  same,  the  rank  orders  of  agreement  and  degree  of  consensus  are  identical. 
Those  nineteen  statements  are  (in  descending  rank  order): 

1.  Within  the  context  of  visual  literacy,  a  viswa/ which  is  visible  may  include  iconic,  pictorial 
representations. 

2.  Visual  literacy  refers  to  the  use  of  visuals  for  the  purposes  of  communication. 
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3.  Within  the  context  of  visual  literacy,  a  visual  may  be  seen  with  the  eyes  (visible). 

4.  Within  the  context  of  vi&ual  literacy,  a  visual  which  is  visible  may  include  man-made 
objects. 

5.  Within  the  context  of  visual  literacy,  a  visual  which  is  visible  may  include  natural  objects. 

6.  Within  the  context  of  visual  literacy,  a  visual  which  is  visible  may  include  events. 

7.  The  processes  humans  engage  in  to  practice  visual  literacy  (use  visuals  for  the  above 
purposes)  include  responding  to  visuals. 

8.  Designing  visuals  includes  using  the  elements  of  visual  design  such  as  color,  line,  fom, 
shape,  light. 

9.  Responding  to  visuals  may  be  done  mentally. 

10.  Using  visuals  in  communication  may  provide  a  means  of  communicating  other  than 
verbally. 

11.  Using  visuals  in  communication  may  enhance  communication  by  combining  visual  and 
verbal  elements. 

12.  Within  the  context  of  visual  literacy,  a  visual  which  is  visible  may  include  actions. 

13.  Designing  visuals  includes  applying  the  principles  of  composition  such  as  balance,  rhythm, 
repetition  and  unity. 

14.  Designing  visuals  includes  using  visual  cues  to  indicate  movement,  passage  of  time, 
distance. 

15.  Perception  of  visuals  includes  attending  to  the  visual  stimuli. 

16.  The  influences  upon  meaning  may  lie  within  the  interactions  among  these  elements 
[within  the  viewer,  within  the  visual  image,  within  the  context  of  the  presentation,  within  the 
context  of  sociological  influences]. 

17.  Visual  literacy  involves  the  affective  domain  in  that  attitudes  and  emotions  can  be  evoked 
with  visuals. 

18.  The  study  o/ visual  literacy  involves  the  relationship  among  the  components  of  research, 
theory,  and  implementation. 

19.  Within  the  context  of  visual  literacy,  a  mental  visual  includes  pictorial  images. 

[Note:  The  complete  list  of  accepted  statements  is  available  to  other 
researchers  when  requested  directly  from  the  authors.] 


Results  of  the  Validation  Phase 

Rating  of  entire  instrument.  A  Hest  was  performed  on  the  summed  means  of  all  the  items 
on  Round  Three  of  the  Delphi  study  and  the  summed  means  of  all  the  items  on  the  final 
administration  of  the  instrument.  The  difference  in  the  mean  responses  was  found  not  to  be 


ERLC 


036 

48 


statistically  significant  (r=1.40,  df=71,  p=.123)  between  the  results  of  the  Delphi  study 
(M=657.33,  n=43)  and  the  final  administration  (M=629.37,  n=30). 


Ratify  of  individual  items.  A  /-test  was  performed  on  each  individual  item  to  identify 
those  statements  which  were  rated  significantly  differently  by  the  two  groups.  There  was 
found  to  be  a  significant  difference  in  means  on  35  of  the  statements.  No  significant 
difference  was  found  in  the  mean  rating  of  151  items. 

Consensus.  An  F-ratio  was  computed  to  determine  if  the  variance  in  the  responses  on 
each  iteration  of  the  Delphi  study  was  significantly  different  from  the  variance  in  the 
responses  on  the  final  administration  of  the  instrument.  The  results  were: 

There  was  found  to  be  no  statistically  significant  difference  (F=1.74,  df=49/29,  p=.085) 
in  the  variance  among  the  responses  to  the  first  iteration  of  the  Delphi  instrument 
(var=5868.94,n=50)  and  the  validation  phase  administration  (var=10,219.59,n=30). 

The  differences  in  the  variance  for  the  instrument  as  a  whole  (all  186  items)  was  found 
not  to  be  statistically  significant  (F=4.05O,  df=42/29,  p<.0005)  between  the  results  of  the 
final  iteration  of  the  Delphi  study  (var=2521.04,n=43)  and  the  validation  administration 
(var=10,219.59,n=30). 

Conclusions 

1.  The  Delphi  technique  proved  to  be  an  appropriate  methodolo^  for  identifying  visual 
literacv  constructs.  The  186  statements  generated  in  the  first  round  yielded  167  statements 
that  survived  the  agreement  level  criterion. 

2.  The  Delphi  technique  proved  appropriate  for  bringing  visual  literacv  experts  to  consensus 
regarding  the  elements,  concepts  and  constructs  of  visual  literacv.  Given  a  set  of  statements 
from  Round  One,  the  participants  significantly  narrowed  their  degree  of  agreement  when 
rating  those  statements  in  the  next  two  rounds. 

3.  The  very  strong  correlation  between  the  level  of  agreement  and  the  degree  of  consensus 
(r=.908)  indicates  that  there  is  the  most  agreement  on  those  constructs  which  are  the  most 
highly  accepted. 

4.  The  set  of  agreed-upon  statements  emanating  from  this  study  can  be  the  basis  for 
structuring  a  coherent  theoretical  base  for  visual  literacv.  No  longer  can  the  charge  be  made 
that  there  is  no  substance  to  visual  literacy  or  that  the  emerging  field  of  visual  literacy  has  no 
conceptual  foundations.  If  VL  appears  to  lack  these  things,  it  is  likely  a  reflection  of  an 
immature  discipline  rather  than  an  unsubstantiated  one.  The  constructs  exist. 

5.  Notvyithstanding  statements  to  the  contrary  in  the  literature,  there  is  a  high  level  of 
agreement  on  most  factors  associated  with  visual  literacv.  Tl.;:  majority  of  parlicipants 
agreed,  strongly  or  moderately,  with  all  186  statements  generated  by  the  first  round  of  the 
study.  Moreover,  167  statements  were  agreed  upon  at  a  statistically  significant  (p=.05)  level. 
These  exceptionally  high  agreement  levels  indicate  that  there  is  indeed  much  agreement 
about  the  constructs  of  visual  literacy    at  least  among  that  segment  of  the  population  who 
consider  themselves  knowledgeable  about  VL.  The  term  'Visual  literacy"  niay  forever  be 
criticized  by  those  who  treat  the  ^'  rds  literally  or  who  themselves  want  to  impose  a 
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definition.  However,  the  term  is  widely  understood  and  the  meaning  given  to  it  is  far  more 
universal  than  the  detractors  imply. 


The  same  is  true  regarding  the  makeup  of  visual  literacy  as  a  field.  The  elements, 
components,  and  theoretical  constructs  are  agreed  upon  at  a  remarkably  high  level  by  those 
individuals  recognized  as  visual  literacy  experts. 

6.  Phase  II  of  the  study  generally  validated  the  findings  of  Phase  L  A  second  group  of 
participants,  using  a  different  criterion  for  rating,  supported  (as  important)  the  VL  constructs 
identified  by  the  Delphi  group.  The  validation  group  ranked  visual  literacy  statements 
similarly  to  the  Delphi  group  for  151  of  186  items. 


Summary 

This  study  was  initiated  to  examine  the  issue  of  apparent  lark  of  agreement 
concerning  what  the  critical  components  of  visual  literacy  are.  Contrary  to  the  conclusion 
that  could  easily  be  drawn  from  a  review  of  the  existing  literature,  this  study  shows  that, 
among  the  scholars  who  study  visual  literacy,  there  appears  to  be  a  great  deal  of  agreement 
regarding  its  basic  tenets. 
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Facilitating  Allied  Health  Students'  Acquisition  cf  Clinical  Reasoning  Skill 


ABSTRACT 

This  paper  discussess  two  methods  used  to  clarify  patients*  problems  and  develop 
treatment  plans  to  resolve  them.  The  traditional  problem  solving  approach  is 
presented  as  a  means  to  introduce  students  to  patient  evaluation  and  treatment 
planning*  while  clinical  reasoning  is  seen  as  a  more  expert  way  to  approach  this 
process.  The  incorporation  of  problem  solving  early  in  allied  health  curricula, 
followed  by  instruction  and  practice  in  clinical  reasoning  is  recommended.  An 
instructional  model  for  facilitating  students'  acquisition  of  clinical  reasoning 
skill  is  presented. 
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The  enhancement  of  students'  ability  to  'problem  solve"  has  received  much 
attention  in  recent  allied  health  literature.  Concurrently,  in  the  medical 
ec  jcation  literature  there  is  a  discussion  of  the  "clinical  reasoning  process,"  a 
process  used  to  think  through  patients'  problems  and  formulate  treatment  plans. 
Both  problem  solving  and  clinical  reasoning  have  as  their  goal  the  determination 
of  a  patient's  problem  and  the  development  of  treatment  plans  to  resolve  the 
problem.  The  purpose  of  this  paper  "s  to  compare  these  two  methods  of  resolving 
patients'  problems,  and  to  suggest  a  method  of  instruction  that  will  facilitate 
students'  acquisition  of  more  expert  reasoning. 

BACKGFOUND 

Problem  Solving  Approaches 

The  discussion  of  problem  solving  in  the  allied  health  professions  has 
centered  around  teaching  strategies  to  enhance  students'  ability  in  this  area 
(Burnett  &  Pierson,  1988;  Olsen,  1983;  May  &  Newman,  1980;  Jenkins, 
1985;  Slaughter,  Brown,  Garner  &  Peritt.  1989)  Although  each  of  these  model? 
for  teaching  problem  solving  vary  slightiy,  essentially  the  approach  provides 
students  with  evaluation  results  which  they  analyze,  synthesize  into  problem 
lists,  and  develop  treatment  programs  to  address  the  patient's  problems.  If 
patients  were  actually  treated  in  this  manner  the  practitioner  would  perform  a 
general  evaluation  on  a  patient  noting  all  normal  and  abnormal  findings  without 
judgement  until  the  data  collection  process  is  complete.  The  practitioner  would 
then  review  the  evaluation  findings,  synthesize  the  results,  break  major 
problems  down  into  subproblems,  formulate  goals  for  treatment,  and  finally 
develop  a  treatment  plan.  Figure  1  shows  the  format  of  this  type  of  clinical 
problem  solving.  Although  most  commonly  reported  in  the  literature  on  teaching 
problem  solving,  apparently  this  method  of  addressing  patients'  problems  is  not 
utilized  by  expert  practitioners    (Norman,  1988). 


Clinical  Reasoning 

Norman  has  noted  that  the  majority  of  problems  that  are  addressed  in  clinical 
medicine  are  solved  by  means  that  do  not  fit  this  typical  definition  of  a  problem 
solving  model  (Norman,  1988).  Instead,  physicians  use  what  has  become  known 
as  a  "clinical  reasoning  process"  (CRP)  in  addressing  their  patients'  problems. 
Barrows  described  CRP  as  involving  the  use  of  hypotheses  to  guide  the  evaluation 
process,  with  diagnostic  questions  or  tests  used  to  confirm  or  refute  the 
hypotheses  (Barrows  &  Peltovitch,  1987). 

According  to  Barrows  the  hypothesis-test  method  of  clinical  reasoning 
involves  generating  a  hypothesis  (or  preferably,  multiple  hypotheses)  regarding 
the  patient'?  problems  early  in  the  evaluation  process.  Early  hypotheses  may  be 
fairly  general,  not  strongly  tied  to  a  specific  diagnosis  or  problem  source.  These 
initial  hypotheses  are  then  mentally  rank  ordered  in  terns  of  their  plausibility. 
The  most  likely  hypothesized  cause  is  then  investigated,  either  with  further 
questioning  of  the  patient  or  with  specific  evaluative  tests-  As  more  information 
is  gathered  through  the  evaluation  the  practitioner  continually  re-orders  the 
hypotheses  in  terms  of  their  plausibility,  deletes  some,  and  gradually  develops  a 
more  clear  understanding  of  the  patient's  problem(s).  Throughout  the  evaluation 
process  the  practitioner  may  rp^ert  to  a  more  general  evaluation  format  to 
gather  more  information  on  proolems  that  should  not  be  overlooked,  eventually 
returning  to  the  hypothesis-test  method  as  the  process  continues.  Figure  2 
depicts  the  hypothesis-test,  clinical  reasoning  process  in  flow  chart  format. 
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While  Barrow's  work  focused  on  the  clinical  reasoning  processes  utilized  by 
physicians,  evidence  exists  that  similar  reasoning  processes  are  used  by 
experienced  physical  therapists  (Payton,  1985),  and  presumably  other  allied 
health  practitioners. 

A  comparison  of  clinical  reasoning  and  problem  solving  shows  is  that  there 
are  two  schools  of  thought  regarding  how  to  address  patients*  problems.  The  first 
(what  we  term  "problem  solving"  approach)  involves  the  collection  of  all 
evaluative  information  witliout  judgement,  followed  by  an  analysts  of  the  data 
collected.  Clinica*  reasoning,  on  the  other  hand,  involves  an  analyze-as-you-go 
approach.  The  results  of  each  evaluative  question  or  test  is  immediately 
interpreted  and  integrated  with  previously  collected  information  to  guide  the 
practitioner  to  the  next  test  to  be  performed  or  question  to  be  asked.  On  one  level 
the  differences  between  these  two  approaches  to  patients'  problems  appears  to  be 
simply  a  difference  in  the  sequence  of  patient  evaluations.  Examined  more 
closely,  clinical  reasoning  can  be  seen  as  requiring  more  complex  cognitive 
processing  of  patient  information  which  can  result  in  a  more  efficient  and 
effective  identification  of  the  cause  of  the  patient's  problem. 

Both  the  problem  solving  and  clinical  reasoning  approach  to  patients 
problems  have  their  advantages  and  disadvantages.  Problem  solving  can  be  seen 
as  a  more  structured  approach,  utilizing  routine  evaluation  formats  to  collect 
information  on  patients.  This  method  may  be  essential  for  persons  with 
relatively  little  experience  in  a  profession,  as  utilizing  a  structured  approach 
leaves  one  less  likely  to  "miss"  key  questions  or  tests.  However,  prtolem  solving 
can  also  be  more  time  consuming  than  clinical  reasoning,  and  resuK  in 
unnecessary  evaluative  procedures  being  performed.  Clinical  reasoning  is  a 
more  expert-like  way  of  approaching  patient  problems,  which  involves  higher 
levels  of  cognitive  processing  of  patient  information,  and  more  rapid  and  efficient 
determination  of  the  patient's  problem.  One  potential  problem  with  using  the 
clinical  reasoning  process  to  evaluate  patients  is  that  inexperienced 
practitioners  (and  students)  may  jump  to  conclusions  about  the  patient's  problem 
and  forget  or  simply  delete  as  unnecessary  some  key  evaluative  tests.  However, 
as  persons  gain  a  more  meanin  ul  and  broader  understanding  of  pathologies  and 
their  appropriate  treatments,  a  J  develop  a  repetoire  of  experience  in  their 
chosen  field,  it  is  less  likely  that  such  errors  will  occur. 

The  models  for  problem  solving  instruction  advocate  introducing  the  model 
early  in  the  educational  program  (Burnett  &  Pierson,  1988;  Olsen,  1983;  May 
&  Newman,  1980;  Jenkins,  1985;  Slaughter.  Brown,  Garner  &  Peritt,  1989). 
This  inital  instruction  provides  students  with  the  structure  needed  to  gain 
experience  with  patient  evaluation  and  treatment  planning.  However,  since  one 
goal  of  allied  health  education  is  to  produce  competent  clinicians,  and  apparently 
expert  practitioners  utilize  a  quite  different  approach  to  solving  patient 
problems  than  that  traditionally  referred  to  as  "problem  solving,"  incorporation 
of  the  more  complex  reasoning  process  into  existing  allied  heauh  curricula 
seems  advisable  (Payton,  1985).  Such  instruction  can  be  sequenced  late  in  the 
educational  program,  after  students  have  gained  initial  clinical  experience  using 
the  problem  solving  approach,  instruction  in  the  performance  of  the  clinical 
reasoning  process  has  the  potential  to  accelerate  learners'  transitions  from 
novice  to  expert  clincian.  The  following  describes  a  method  of  instruction  in 
clinical  reasoning  that  could  be  incorporated  inio  educational  programs, 
sequenced  after  students  have  gained  experience  with  the  more  traditional 
problem  solving  approach. 
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INSTRUCTIONAL  MODEL  FOR  CRP 

While  there  have  been  several  reports  of  teaching  strategies  for  developing 
problem  solving  abilities:  little  has  been  written  concerning  the  instructional 
strategies  that  will  facilitate  acquisition  of  clinical  reasoning  skill.  Instruction 
in  the  clinical  reasoning  process  is  made  mnre  difficult  because  it  does  not 
involve  a  linear  type  of  thought  process,  but  ''ather  uses  an  Iterative  approach  to 
patient  evaluation.  Thus,  instruction  on  this  method  of  addressing  patients' 
problems  must  convey  the  complexities  of  the  clinical  reasoning  process  while 
providing  for  practice  of  the  skill,  thereby  enhancing  students'  ability  to  use  the 
CRP  in  their  patient  care. 

While  the  ciinica!  reasoning  process  has  been  described  as  being  distinctly 
different  from  traditional  problem  solving,  it  is  not  so  different  from  the  type  of 
reasoning  used  when  encountering  everyday  un\nowns.  Wright  described 
''diagnostic  induction"  as  being  the  kind  of  reasoning  process  that  is  basic  to 
normal  human  functioning  (Wright.  1989).  This  is  a  process  by  which,  when 
confronted  with  a  "problem"  or  question  to  which  there  is  no  blatantly  apparent 
answer  one  generates  multiple  hypotheses  which  are  ordered  in  terms  of  their 
relative  plausability.  The  plausability  rankings  provide  guidance  on  which 
''leads"  to  follow  when  searching  for  other  relevant  Information  which  will  help 
to  clarify  the  question  or  problem  under  study. 

A  common  example  of  diagnostic  inductive  thinking  is  he  way  one  would 
approach  the  problem  of  a  car  that  won't  start.  Neither  u  mechanic  nor  an 
average  car  owner  would  try  to  determine  the  car's  problem  by  completely 
evaluating  all  of  its  systems  followed  by  a  sorting  out  of  this  evaluation's  results 
to  identify  the  problem.  Instead,  both  the  expert  mechanic  and  the  average  car 
owner  would  make  some  guesses  (or  hypotheses)  as  to  what  was  wrong  with  the 
vehicle,  such  as  the  car  being  out  of  gas.  the  battery  being  dead  because  the  lights 
were  left  on.  or  some  general  problem  with  the  ignition  system.  Based  upon 
these  hypotheses  one  would  perform  some  limited  tests  to  determine  v/hich  one  is 
the  real  problem  (looking  at  the  gas  gauge,  checking  the  light  switch,  etc),  and 
generate  some  plan  to  fix  the  problem.  Showing  students  how  the  CRP  is  similar 
to  the  way  they  already  think  about  common  problems  by  using  an  example  such 
as  this  may  help  to  orient  them  to  the  usefulness  of  a  hypothesis-tesl  method  of 
evaluating  patients. 

To  help  students  understand  the  differences  between  the  CRP  and  problem 
solving  approaches  it  may  be  helpful  to  explicitly  point  out  these  differences. 
Following  this  introduction  to  the  CRP,  examples  of  its  use  in  clinical  practice 
should  be  provided  to  the  learners.  Since  the  CRP  is  actually  performed 
mentally,  without  any  overt  sign  of  the  information  processing  which  occurs, 
presenting  examples  of  it's  use  takes  the  form  of  the  instructor  "thinking  aloud," 
by  talking  students  through  the  steps  involved  in  clinical  reasoning.  When 
presenting  examples  of  clinical  reasoning,  a  hypothetical  patient  case  may  be 
presented  to  the  learners,  with  very  little  detail  provided  at  the  outset.  The 
instructor  would  tl.en  talk  through  the  steps  being  used  (i.e.  the  steps  shown  in 
Figure  2)  by  generating  lists  of  hypotheses  about  the  patient,  rank  ordering  the 
possibilities  according  to  their  plausibility,  deciding  upon  information  needs, 
selecting  appropriate  tests,  receiving  test  results,  modifying  hypotheses,  and  so 
on.  This  method  of  instruction,  based  upon  social  learning  theory,  is  referred  to 
as  modeling  of  the  desired  behavior. 

An  option  to  having  the  instructor  simply  talk  through  the  reasoning  process 
on  a  hypothetical  patient  is  to  use  a  videotape  of  an  actual  patient  evaluation,  with 
the  instructor  narrating  the  thought  processes  used  to  progress  through  the 
evaluation.  Interactive  videos,  which  would  allow  selective  branching  to 
different  segments  of  an  evaluation,  provide  an  excellent,  though  expensive, 
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medium  for  depicting  the  clinical  reasoning  process  in  action.  A  less  expensive 
alternative  which  still  allows  a  greater  touch  of  realism  to  this  demonstration  of 
CRP  is  the  use  of  slides  which  depict  the  various  evaluation  steps,  again  narrated 
by  the  instructor. 

In  whatever  medium  the  example  is  presented  to  the  learners,  the  patient 
case  study  should  be  selected  carefully.  It  should  require  all  of  the  steps  involved 
in  CRP,  yet  be  simple  enough  so  as  to  avoid  requiring  a  larger  knowledge  base 
than  that  possessed  by  the  learners.  One  of  the  complexities  of  teaching  the 
clinical  reasoning  process  (or  any  of  the  problem  solving  approaches)  is  that  the 
success  of  a  student's  reasoning  is  dependant  upon  prior  knowledge  upon  which 
the  reasoning  is  based.  For  example,  one  could  not  expect  a  student  to 
demonstrate  competent  clinical  reasoning  when  evaluating  a  patient  with  a  head 
trauma  if  the  student  did  not  know  the  basics  of  neurology,  and  the  necessary 
evaluation  techniques.  Therefore  it  is  important  that  the  examples  and  practice 
cases  presented  be  based  upon  some  clinical  problem  with  which  the  learners  are 
familiar. 

Following  this  initial  demonstration  of  clinical  reasoning,  students  should  be 
provided  with  opportunities  to  practice  using  the  CRP  on  patient  cases.  Three 
levels  of  practice  that  can  be  used  to  promote  acquisition  of  clinical  reasoning 
skills  include  participative  modelling,  individual  practice  on  case  studies,  and 
practice  in  clinical  education  settings. 

Participative  Modelling 

In  participative  modelling  the  instructor  presents  a  group  of  students  with 
initial  patient  information  and  requests  their  input  on  potential  causes  of  the 
patient*s  problem,  ranking  of  the  hypothesized  causes  etc.  Throughout  this 
process  the  instructor  may  prompt  students  with  questions,  cueing  them  toward 
particular  types  of  information  that  might  be  helpful  in  determining  the  patient*s 
problem.  In  this  way  the  students  get  immediate  feedback  about  the  use  of  CRP  in 
a  non-threatening,  group  environment.  In  participative  modeling  the  instructor 
provides  progressively  less  and  less  assistance  in  performing  the  desired  skill, 
thus  allowing  learners  to  increase  their  competence  and  self-efficacy  in  using 
the  newly  learned  skill.  This  type  of  practice  can  be  used  effectively  in  the  early 
stages  of  learning  a  new  skill. 

Individual  Practice 

After  students  demonstrate  an  understanding  of  this  "new"  type  of  clinical 
reasoning  they  can  progress  to  the  second  level  of  practice.  Here  the  students 
should  be  provided  with  opportunities  to  practice  and  receive  feedback  on  their 
own  individual  use  of  the  process.  Practice  items  for  the  clinical  reasoning 
process  should  take  the  form  of  realistic  patient  case  studies,  much  like  those 
presented  in  the  example.  The  type  of  case  study  to  be  used  for  practice  of  CRP 
should  oe  designed  so  that  practice  closely  parallels  the  use  of  the  process  in 
clinical  settings.  Thus,  all  information  about  a  patient  must  not  be  provided  to 
the  learnar  at  the  outset,  instead,  case  studies  should  begin  with  a  small  amount 
of  information  about  the  patient,  such  as  the  type  of  information  that  would  be 
obtained  when  a  patient  is  asked  "Why  did  you  schedule  an  appointment?" 
Additional  information  about  the  case  study  patient  should  be  provided  to  the 
learner  only  upon  request,  as  this  allows  mimicking  the  question-answer  format 
of  patient  interviews  and  the  test-results  format  of  objective  testing.  Thus,  if  a 
student  fails  to  remember  to  "ask"  a  key  question,  or  "order"  a  vital  test  the 
student  fails  to  receive  crucial  information  that  may  affect  the 
diagnostic/treatment  process. 
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Given  the  iterative  format  of  clinical  reasoning,  the  format  of  practice  items 
provides  some  challenges  to  the  instructor.  While  it  may  seem  awkward,  written 
case  studies  can  be  used  to  provide  this  type  of  practice  case.  For  example,  two 
students  could  work  together,  one  as  the  practitioner,  and  the  other  as  the 
patient,  or  information  source.  The  practioner  would  receive  initial  information 
regarding  the  patient,  and  based  upon  this  information,  would  list  possible  causes 
of  the  patient  problem,  and  rank  order  these  causes  in  terms  of  their  probability. 
The  student  would  then  request  additional  information  from  the  patient  to  confirm 
or  refute  the  most  probable  cause  of  the  problem.  The  type  of  information 
requested  could  be  answers  to  questions,  or  objective  test  results.  The  second 
student  provides  the  practitioner  with  only  the  information  requested,  eventually 
culminating  in  the  practitioner's  selection  of  the  patient's  "diagnosis"  and 
formulation  of  a  treatment  plan. 

This  method  of  working  through  case  studies  is  certainly  feasible,  but  is 
dependant  upon  the  ability  of  learners  to  work  together  in  teams.  Computer 
programs  written  to  provide  patient  information  only  upon  request  could  be  used 
to  allow  individual  student  practice.  Interactive  videos  programmed  to  provide 
information  to  the  learners  in  the  same  manner  could  also  present  an  effective 
and  powerful  method  for  practicing  CRP,  this  time  allowing  the  learners  to 
practice  observation  skills  that  are  not  afforded  by  the  written  or  non-video 
computer-based  practice  methods. 

In  addition  to  the  format  of  practice  items  it  is  also  important  to  consider  the 
sequence  in  which  they  are  presented  to  students.  Easy  to  difficult  sequencing  is 
recommended  so  that  learners  may  first  practice  on  problems  on  which  they  have 
a  relatively  high  probability  of  successfully  completing,  thus  allowing  them  to 
practice  a  new  skill  in  low  stress  situations.  Easy  cases  might  describe  patients 
whose  problem  is  easily  identifiable,  has  a  fairly  obvious  source,  or  is  commonly 
seen  in  clinical  settings.  More  difficult  cases  might  include  patients  with 
multiple  problems,  or  seemingly  contradictory  evaluation  results.  The  use  of 
divergent  practice  items  is  also  recommended,  to  allow  students  to  practice 
applying  the  CRP  with  different  types  of  patient  problems. 

During  this  level  of  practice  it  is  essential  that  students  receive  corrective 
feedback  regarding  the  effectiveness  of  their  clinical  reasoning.  Feedback  from 
the  instructor  should  concentrate  not  only  on  the  outcomes  of  the  learner's  use  of 
CRP  (i.e.  the  identified  patient  problem  and  treatment  plan),  but  also  on  the 
process  itself,  perhaps  by  pointing  out  specific  steps  on  which  the  learner  needs 
to  focus.  When  using  the  CRP  two  types  of  errors  can  be  made.  First,  students 
may  use  the  CRP  itself  incorrectly,  referred  to  as  a  process  error.  For  example, 
when  presented  with  initial  patient  information  a  student  may  "collect"  data 
randomly,  without  using  hypothesized  causes  of  the  patient's  problem  to  guide  the 
data  cclloction  process.  This  type  of  error  might  be  identified  by  asking  the 
student  why  they  elected  to  "perform"  a  specific  test,  and  what  kind  of 
information  they  hoped  to  gain  from  it. 

The  second  type  of  error  that  can  occur  when  using  the  CRP  is  making 
incorrect  assumptions  about  the  meaning  of  test  results,  or  the  symptoms 
exhibited  by  a  patient  with  a  certain  problem.  These  errors  can  be  classified  as 
knowledge  errors.  Students  working  with  this  type  of  error  might  appear  to  be 
randomly  collecting  data  on  a  patient,  but  actually  be  working  with  a  deficient 
knowledge  base,  and  thus  be  generating  implausible  hypotheses.  This  type  of 
error  requires  identification  of  the  student's  incorrect  assumptions,  and  perhaps 
referral  to  text  books  or  other  references  for  remedial  work. 

When  providing  feedback  about  students'  clinical  reasoning  skill,  it  is 
important  to  remember  that  no  two  individuals  can  be  expected  to  take  the  same 
paths  through  the  reasoning  process  on  any  given  patient.  The  CRP  is 
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individualistic  in  that  many  of  the  steps  require  judgements  based  upon  prior 
knowledge  and  experience.  For  example,  when  presented  with  a  patient  with  low 
back  pain,  one  physical  IhGrapisl  may  rank  as  tho  most  probabls  hypothesized 
cause  of  the  pain  as  being  a  generated  from  a  disc.  An  equally  competent  therapist 
with  different  prior  experiences  may  believe  that  the  most  probable  cause  is  a 
muscle  sprain,  and  still  another  might  hypothesize  facet  involvement.  From 
these  diverse  starting  points  each  therapist  will  progress  by  asking  questions  and 
performing  tests  to  confirm  their  own  hypothesis,  and  eventually  revising  their 
hypothesis  to  keep  it  in  line  with  the  Information  gathered  from  the  evaluation. 

Since  one  can  not  expect  two  experts  to  follow  the  same  paths  in  thinking 
through  a  patient's  problem  it  is  important  that  one  does  not  expect  all  students 
to  perform  the  reasoning  in  exactly  the  same  manner.  The  important  point  is 
that  students  do  not  make  critical  errors  in  their  clinical  judgement,  such  as 
forgetting  key  tests  that  might  differentiate  one  type  of  patient  problem  from 
another.  To  help  students  recognize  how  important  it  is  both  to  consider  all 
possible  causes  of  a  patient's  problem,  and  to  not  jump  to  conclusions  about  the 
problem  source  early  in  the  evaluation  process,  contrasting  patient  cases  should 
be  included  in  the  practice  of  GRP.  This  involves  the  development  of  cases  in 
which  the  patients  have  similar  presenting  symptoms  which  result  from 
different  causes.  A  physical  therapy  example  of  these  type  of  cases  would  be  to 
present  students  with  a  case  of  a  patient  with  shoulder  bursitis,  and  one  with  a 
cervical  radiculopathy.  Both  patients  could  have  similar  (though  not  identical) 
histories  of  the  onset  of  the  problem,  identical  pain  distributions,  and  very 
similar  range  of  motion  and  muscle  strength.  However,  a  few  key  tests  could  be 
performed  on  such  patients  to  differentiate  between  the  two  problems.  Failing  to 
perform  the  tests  might  result  in  the  incorrect  identification  of  the  patient 
problem,  and  development  of  an  inappropriate  treatment  plan. 

Clinical  Education  Practice 

The  final  level  of  practice  of  the  clinical  reasoning  process  should  occur  in 
the  most  realistic  setting  possible.  Clinical  education  experiences  are  ideal 
forums  for  practice  of  clinical  reasoning,  for  it  is  in  the  dinic  that  the  process 
takes  on  real  meaning  for  the  students  and  their  patients.  Under  the  supervision 
of  clinical  preceptors  students  can  further  refine  their  clinical  reasoning  skills, 
and  experience  first  hand  the  results  of  their  reasoning.  Following  a  student 
conducted  patient  evaluation  the  clinical  preceptor  can  use  in  depth  questioning  of 
the  student  to  detemnine  why  certain  tests  were  or  were  not  performed.  This 
will  assist  the  preceptor  in  understanding  the  student's  clinical  reasoning,  and 
will  help  to  identify  weaknesses  in  process  or  knowledge  that  might  lead  to 
ineffective  determination  of  the  patient's  problem.  The  preceptor  might  then 
suggest  potential  causes  of  the  patient's  problem  lhat  the  student  did  not  consider, 
and  correct  any  faulty  assumptions  that  the  student  might  have  made  during  the 
evaluation. 

DISCUSSION 

The  reasoning  process  utilized  by  health  professionals  is  a  complex  one, 
which  requires  integration  of  information  from  numerous  sources.  The  clinical 
reasoning  process  presented  here  can  be  seen  as  distinctly  different  from 
traditional  problem  solving  models.  The  model  presented  for  CRP  instruction  is 
intended  to  clarify  the  clinical  reasoning  process  itself,  and  to  identify 
instructional  strategies  to  assist  students  in  acquiring  this  critical  skill.  Figure 
3  depicts  the  steps  described  for  instruction  on  clinical  reasoning. 

While  practical  for  implementation  in  most  allied  health  educational 
programs,  this  method  for  teaching  the  clinical  reasoning  process  has  not  been 
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subjecied  to  testing  with  students  to  determine  it's  effectiveness  in  achieving  the 
goal  of  acquisition  of  more  expert-like  GRP  skills.  One  difficulty  in  testing  any 
insiruciionai  meihod  for  ciinicai  reasoning  is  ihe  difficulty  in  measuring 
performance  outcomes.  The  authors  know  of  no  tool  which  altows  for  assessment 
of  students'  ability  to  reason  in  clinical  settings,  save  clinical  preceptor's 
subjective  assessment  of  student  performance.  A  recent  effort  to  determine  the 
effectiveness  of  a  clinical  problem  solving  model  through  tht  use  of  a  standard 
critical  thinking  scale  was  unsuccessful,  perhaps  not  so  much  because  of  the 
ineffectiveness  of  the  model,  but  because  the  testing  instrument  used  might  not  be 
an  effective  measure  of  the  problem  solving  used  in  allied  health  practice  5  The 
identification  of  an  effective  evaluative  instrument  for  measuring  learners' 
clinical  reasoning  skill  would  be  of  great  assistance  in  determining  the  value  of 
any  clinical  reasoning  instructional  model. 

As  mentioned  previously,  the  use  of  the  clinical  reasoning  process  is 
dependant  upon  the  student's  prior  knowledge,  and  their  ability  to  recall  and 
integrate  information  from  prior  coursework  that  might  assist  in  the  GRP. 
Current  work  in  instructional  design  related  to  schema  theory  may  help  to 
identify  ways  lO  design  instruction  so  that  this  information  is  more  easily 
retreivable  for  use  in  the  clinical  setting.  One  problem  that  we  have  observed  in 
students  beginning  to  evaluate  patients  in  clinica!  settings  is  their  inability  to 
distinguish  between  relevant  and  irrelevant  evaluation  findings.  Using  the 
clinical  reasoning  process  may  help  to  reduce  the  amount  of  irrelevant 
information  that  students  obtain  from  patient  evaluations,  as  each  test  or 
question  is  related  to  their  hypothesis  regarding  the  cause  of  the  patient's 
problem.    Again,  instructional  strategies  that  will  facilitate  identification  of 
relevant  r'»es  would  be  of  assistance  in  preparing  students  for  competent  clinical 
practice. 

Additional  research  into  the  area  of  clinical  reasoning  may  be  directed  toward 
the  identification  of  critioai  characteristics  of  case  studies  to  be  used  in  the 
practice  of  GRP.  Investigation  into  the  attributes  of  media  through  which  GRP 
practice  is  provided  may  also  provide  guidance  in  the  design  of  effective  learning 
experiences  in  this  area. 

Another  area  of  interest  is  the  issue  of  sequencing  clinical  reasoning 
instruction.  We  have  recommended  introduction  of  "problem  solving"  early  in 
the  health  care  professionals'  curricula  as  a  means  of  providing  learner 
motivation  and  initial  structuring  of  knowledge  for  use  in  health  care  settings. 
This  initial  instruction  would  be  followed  by  later  instruction  in  the  hypothesis- 
test  method  of  GRP.  The  question  of  whether  it  is  necessary  or  advisable  to  teach 
problem  solving  before  hypothesis-test  has  not  been  tested  empirically.  Some 
learners  may  be  able  to  generate  sufficient  numbers  of  hypotheses  regarding 
patient  problems  early  in  the  curriculum  so  that  simple  problems  may  be 
addressed  through  this  more  expert-like  method.  On  the  other  hand,  other 
learners  may  not  be  able  to  utilize  the  hypothesis-test  method  until  they  have 
completed  extensive  clinical  experience  in  their  chosen  fields,  and  may  not  use 
this  method  until  they  have  completed  their  formal  schooling.  Studies  of 
individual  differences  in  ability  to  generate  and  test  hypotheses  may  provide 
insight  into  the  optimal  sequencing  of  GRP  instruction  (i.e.  problem  solving 
prior  to  the  GRP;  focusing  solely  on  GRP,  or  using  some  other  method  not  yet 
addressed)  for  individual  learners. 
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Figure  1.  The  problem-solving  process.  Problem  solving  is  presented 
as  a  linear  process. 
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Figure  2  Tho  chntcal  reasoning  process  Clinical  roasoning  is  an  iterative  proctiss  In  which  information  from  each  evaluative  procedure  affects  the  selection 
of  the  next  procedure 
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Figure  3  A  model  for  instruction  in  clinical  reasoning  Note  that  this  instruction 
builds  upon  a  foundation  of  both  coursework  and  prior  experience  with  problem  solving 
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Duririg  the  past  several  years,  extensive  research  has  addressed  the 
interaction  between  hemispheric  specialization  and  the  processing  of  visual 
information.  Studies  suggest  that  visual  material  may  be  stored  in  memory 
in  different  locations  or  forms  depending  upon  how  meaningful  the  visual 
cues  are  in  relation  to  the  information  presented.  Other  researchers  have 
explored  the  degree  of  processing  of  visual  information  in  the  right  and  left 
cerebral  hemispheres.  Little  research  however,  has  investigated  the  specific 
interaction  l^tween  how  color  information  is  processed  and  the  relationship 
of  that  processing  to  hemispheric  specialization. 

The  purpose,  therefore,  of  this  study  was  to  investigate  the  interaction 
between  hemispheric  specialization  and  pictorial  recognition  memory  for 
pictures  presented  in  three  different  color  modes:  realistic  color,  non-realistic 
color  and  monochrome  (black  and  white).  A  secondary  purpose  was  to 
further  confirm  the  efficacy  of  applying  signal  detection  analysis  to  color 
recognition  memory  data  as  a  means  of  obtaining  a  more  accurate  assessment 
of  the  role  of  color  in  visual  information  processing. 


BACKGROUND  RESEARCH 

Substantial  research  has  focused  on  the  role  of  color  in  visualized 
instruction  (Dwyer,  1972, 1978;  Berry,  1974;  Winn,  1976;  Chute,  1979; 
Lamberski,  1980).  This  research  represents  one  aspect  of  the  larger  theoretical 
debate  which  continues  regarding  visual  complexity  and  hiiman  information 
processing.  It  has  long  been  contended  that  the  mere  addition  of  visual  cues 
will  inaease  the  ability  of  the  viewer  to  store  ar»d  retrieve  visual 
information..  This  orientation,  termed  "realism  theory"  by  Dwyer  (1967),  has 
strong  theoretical  foundations  (Dale,  1946;  Morris,  1946;  Carpenter,  1953  and 
Gibson,  1954)  and  is  indeed  the  major  premise  of  cue  summation  theory 
(Severin,  1967).  Other  researchers  (Broadbent,  1958,  1965;  and  Travers,  1964) 
have,  however,  taken  strong  opposition  to  this  theoretic  base  on  the  grounds 
that  the  human  information  processing  system  is  of  limited  capacity  and  that, 
in  times  of  rapid  information  reception,  irrelevant  cues  may  block  the 
processing  of  other,  relevant  information.  Studies  (Kanner,  1968;  Katzman 
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and  Nyenhuis,  1972;  Dwyer,  1972, 1979)  have  investigated  this  apparent 
contradiction  with  conflicting  results. 

The  inclusion  or  absence  of  color  information  can  be  regarded  as  one 
dimension  of  visual  complexity.  Color  can  function  in  a  dual  role  when  used 
in  visual  displays.  First,  it  can  serve  primarily  a  coding  function,  providing 
additional  information  but  not  providing  any  realistic  desaiption  of  the 
display.  In  this  case,  the  effectiveness  of  color  can  be  predicted  by  cue 
summation  theory,  but  not  by  the  realism  hypothesis.  Alternately ,  color  can 
be  used  to  present  a  mere  realistic  version  of  the  visual  display.  In  this 
instance,  in  addition  to  providing  a  greater  number  of  overall  cues,  it 
provides  the  viewer  with  more  realistic  attributes  or  "handles"  with  wliich  to 
store  and  retrieve  information.  When  color  is  used  in  this  fashion,  its  value 
could  be  predicted  by  the  realism  theories  as  well  as  by  cue  summation  theory. 

Much  past  research  investigating  the  differences  between  color  and 
monochrome  visuals  f  iled  to  take  into  account  the  fact  that  realistic  color 
visuals  contain  intrinsically  more  information  and  consequently  require 
more  time  for  processing.  In  an  attempt  to  resolve  this  methodological 
inconsistency  as  well  as  to  more  accurately  assess  the  role  of  color  in  human 
information  processing.  Berry  (1974)  compared  realistic  and  non-realistic 
color  versions  of  the  instructional  materials  on  the  human  heart  developed 
by  Dwyer  (1967).  Data  suggested  that,  in  those  learning  tasks  where  visual 
materials  contributed  significantly  to  the  improvement  of  instruction, 
realistic  color  materials  were  most  effective.  Later  research  (Berry,  1977, 1982, 
1983)  which  investigated  the  color  realism /complexity  question  relative  to 
pictorial  recognition  memory  found  both  realistic  and  non-realistic  color 
materials  superior  to  monochrome  visuals.  These  findings  suggest  that  cue 
summation  theory  may  provide  an  accurate  desaiption  of  how  color 
functions  in  basic  information  processing  tasks  such  as  picttire  recognition. 

Other  research  by  Berry  (1984),  in  which  the  role  of  the  cognitive  style 
of  field  dependence  identified  by  Witkin  et  al  (1971)  was  investigated  in 
relation  to  color  information  processing,  suggested  that  field  dependent 
individuals  experience  greater  processing  and  storage  difficulties.  In  this 
study,  it  was  concluded  that  such  difficulties  may  represent  less  efficient 
organization  of  the  material  in  memory. 

Studies  by  Craik  and  Lockhart  (1972)  and  Craik  and  Tuiving  (1975) 
strongly  suggest  that  a  continuum  of  levels  of  information  processing  exists 
and  that  more  meaningful  information  is  processed  more  deeply  and  thereby 
is  more  effectively  stored  in  memory.  Such  a  theory  would  imply  that 
realistic  color  materials,  due  to  their  greater  number  of  meaningful  cues 
would  be  more  deeply  processed  and  therefore  more  effectively  stored.  Paivio 
(1971,  1975)  hypothesized  a  dual  coding  theory  in  which  one  system,  the 
imagery  system  processes  information  related  to  conaete  objects  and  events, 
while  the  second,  verbal  system  processes  language-based  iriormatior\.  This 
theory  may  imply  that  more  realistic  information  which  might  be  easily 
described  verbally  could  be  processed  separately  from  that  which  has  fewer 
verbal/realistic  labels.  Much  of  this  research  suggests  that  visual  information 

057 


69 


3 


may  indeed  be  processed  in  a  variety  of  ways  or  in  more  than  one  memory 
location. 

Extensive  research  during  the  past  thirty  years  has  addressed  ttie 
concept  of  cerebral  laterality  and  hemispheric  specialization  as  another  means 
of  explaining  variations  in  the  processing  of  information.  In  summarizing 
this  mass  of  research  findings,  Hellige  (1980)  presents  the  common 
conclusion  that  the  right  hemisphere  of  the  brain  specializes  primarily  in 
visuospatial  processing  and  the  left  hemisphere  focuses  on  verbal/linguistic 
tasks.  While  it  would  be  abStirdly  simplistic  to  attribute  the  processing  of 
specific  types  of  information  to  one  or  the  other  hemispheres  of  the  brain,  it 
would  not  be  unreasonable  to  expect  visual  information  to  be  processed 
somewhat  differently  in  the  different  locations.  Although  it  is  generally  held 
that  visual  information  is  processed  first  in  the  right  hemisphere,  it  is  not 
clear  how  or  where  color  information  is  processed.  Beauvois  and  Saillant 
(1985)  however,  found  significant  data  which  indie:  led  that  color  is 
represented  in  two  separate  memory  systems,  one  verbal  and  one  visual. 
When  color  is  used  in  a  realistic  manner,  it  cari  supply  additional, 
meaningful  verbal  or  linguistic  labels,  while  the  use  of  color  simply  as  a 
coding  technique  (non-realistic)  may  not  be  processed  in  the  left  hemisphere. 
Research  by  Meyer  (1975),  Daehler  et  al  (1976)  and  Lamberski  and  Dwyer  (1983) 
support  the  coding  concept  of  color. 

Research  conducted  on  brain  damaged  subjects  by  De  Renzi  and 
Spinnler  (1967)  found  that  subjects  with  right  hemisphere  damage  had 
significantly  more  pjoblerns  in  color  naming  and  identification  tasks  while 
subjects  with  left  hemisphere  lesions  did  not  demonstrate  significant 
impairment.  These  results  were  further  confirmed  in  studies  by  Scotti  and 
Spinnler  (1970),  and  Capitani,  Scotti  and  Spinnler  (1978).  A  study  conducted 
by  Davidoff  (1976)  with  normally  functioning  subjects  reported  a  right 
hemisphere  advantage  in  sensitivity  to  perception  of  color  stimuli.  These 
findings  were  confirmed  in  a  later  study  (Davidoff,  1977).  Malone  and 
Hannay  (1978)  however,  investigated  color  memory  relative  to  hemispheric 
asymmetry  in  normal  subjects.  Results  indicated  a  significant  main  effect  for 
color  in  the  right  visual  field,  implying  a  left  hemisphere  advantage.  The^e 
results  were,  however,  attributed  to  the  use  of  verbalization  in  the 
performance  task.  Jorgenson,Davis,  Opella  &  Angersiein  (1980,  1981)  found  a 
right  hemisphere  advantage  for  color  and  a  left  hemisphere  advantage  for 
color/verbal  responses. 

From  the  preceding  review,  it  is  apparent  that  the  means  by  which 
color  information  is  processed  is  still,  as  Otto  and  Askov  (1968)  concluded, 
essentially  unclear.  It  was  the  purpose  therefore,  of  this  study  to  investigate 
differences  in  the  processing  of  color  information  within  the  two 
hemispheres  of  the  brain.  Since  it  is  a  well  established  fact  that  the  two 
hemispheres  do  not  function  independently,  it  would  be  inappropriate  to 
attribute  specific  modes  of  processing  to  one  or  the  other  discreet 
hemispheres.  Consequently,  in  this  study,  comparisons  were  made  between 
the  processing  attributable  to  the  integrated  functioning  of  both  hemispheres 
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as  is  normally  the  case,  with  processing  accomplished  by  the  right  hemisphere 
independent  of  the  left.  To  achieve  this  condition,  it  was  necessary  to  occupy 
the  left  hemisphere  with  a  complex  verbal  m.asking  task,  thereby  effectively 
preventing  it  from  effectively  processing  the  presented  visual  information. 
Such  a  technique  has  become  a  commonly  accepted  means  of  localizing 
processing  to  the  right  hemisphere. 


METHOD 

The  stimulus  materials  used  in  the  study  were  similar  to  those  used  by 
Berry  (1977, 1982, 1983, 1984)  ar.d  EI  Gazzar  (1984).  These  consisted  of  120 
stimulus  slides  and  60  distractor  slides.  All  slides  were  obtained  from  a  pool 
of  geographic  scenery  slides  taken  by  semi-professional  and  professional 
photographers  in  various  locations  of  the  United  States  and  Cai\ada.  In 
selection  of  the  materials,  care  was  exercised  to  exclude  all  recognizable 
human  figures,  verbal  materials  or  unique  objects.  The  entire  collection  of 
materials  was  randomly  divided  into  thirds.  One  third  was  retained  as  a 
realistic  color  group,  a  second  third  was  recopied  onto  black  and  white  slides 
and  the  remaining  third  was  altered  by  photographic  reversal  to  produce  a 
non-realistic  color  group.  By  means  of  the  photographic  reversal  process,  the 
overall  number  of  color  cues  could  be  held  constant,  while  the  degree  of  color 
realism  could  be  manipulated. 

The  sample  for  the  study  consisted  of  twenty  six  graduate  students 
drawn  from  academic  areas  of  education,  health  professions  and  library 
science.  All  subjects  were  consenting  volunteers. 

The  list  learning  procedure  was  employed,  in  which  all  subjects  were 
first  shown  the  set  of  120  stimulus  slides,  individually  for  500  ms  each.  On  a 
random  basis,  subjects  were  instructed  to  perform  a  masking  task  while 
viewing  either  the  first  sixty  slides  or  the  second  sixty  slides.  The  remaining, 
alternate  sixty  slides  were  viewed  without  the  performance  of  the  masking 
task.  The  task  employed  required  each  subject  to  begin  counting  backward  by 
threes  from  the  number  488  as  quickly  as  possible,  while  viewing  the  specified 
group  of  stimulus  slides.  In  this  way,  right  hemisphere  processing  could  be 
localized  by  inhibiting  left  hem.isphere  processing.  Subjects  were  subsequently 
presented  with  a  random  distribution  of  all  slides  (stimulus  and  distractor) 
for  five  seconds  each.  During  that  time,  subjects  responded  on  a  checklist 
either  "old"  (stimulus  slide  -  seen  before)  or  "new"  (distractor  slide  -  never 
seen). 

The  design  of  the  study  was  a  two-way  repeated  measures  type  with 
three  levels  of  the  color  factor  (realistic,  non-realistic,  monochrome)  and  two 
levels  of  the  cerebral  localization  factor  (right  specialized,  integrated). 
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ANALYSIS 

/\naiySl5  or  tiie  uctid  uuiaineu  was  i.uiiuu«.icu  via  itte  »»ici»iou  L^igjinSi 

Detection  Analysis.  This  approach  has  been  frequently  applied  to  the  analysis 
of  recognition  data  in  the  past  as  a  means  whereby  both  recognition  rate  and 
error  rate  are  taken  into  account. 

Signal  Detection  Theory  has  been  accepted  as  a  reliable  technique  for 
assessing  a  subject's  ability  to  desaibe  the  occurrence  of  disaete  binary  events. 
The  basic  model  of  SDT  was  desaibed  by  Swets  (1964)  and  has  been  used 
extensively  to  study  the  ability  of  individuals  to  distinguish  the  presence  of  a 
signal  when  that  signal  was  mixed  with  noise.  More  recently,  Grasha  (1970) 
has  suggested  the  use  of  SDT  parameters  in  the  study  of  memory  processes. 
Signal  Detection  Theory  has  been  applied  specifically  to  recognition  memory 
experiments  involving  pictorial  material  in  research  conducted  by  Snodgrass, 
Volvovitz  and  Walfish  (1972),  Loftus  and  Kallman  (1979),  Loftus,  Greene  and 
Smith  (1980),  Berry  (1982, 1983, 1984)  and  El  Gazzor  (1984). 

The  mean  probability  of  hits  and  the  mean  error  rates  for  each 
treatment  and  level  were  calculated  as  well  as  the  measure  of  sensitivity,  d', 
which  was  determined  from  tables  developed  by  Elliot  (1964).  To  facilitate 
analysis,  the  d'  statistic  was  uniformly  tiransformed  to  eliminate  negative 
values  (d'  adjusted).  These  data  a;   presented  in  table  1. 

TABLE  1 

Means  and  standard  deviations  for  hit  and  false  alarm  probabilities, 
error  rates  and  d'  by  treatments  and  cerebral  localization 
(N  =  26) 


P(H)  P(FA)  ERRORS      d'  ADJUSTED 


TREATMENT 

Mean 

s.d. 

Mean 

S.d. 

Mean 

S.d. 

Mean 

s.d. 

Right  localized 

Realistic  Color 

.480 

.163 

.519 

.134 

16.56 

2.98 

2.918 

.431 

Nonrealistic  Color 

.565 

.193 

.419 

.176 

13.07 

3.34 

3.427 

.487 

Black  &  White 

.479 

.184 

.492 

.136 

14.12 

3.32 

2.909 

.411 

Integrated 

3.24 

3.283 

.486 

Realistic  Color 

.617 

.174 

.519 

.134 

13.87 

Nonrealistic  Color 

.533 

.181 

.419 

.176 

13.75 

2.90 

3.236 

.555 

Black  &  White 

.546 

..203 

.492 

.136 

12.77 

3.73 

3.143 

.579 

Analysis  of  variance  procedures  for  two  factor  repeated  measures  were 
conducted  on  the  probability  of  hits  (recognition  scores),  total  error  scores  and 
on  the  adjusted  SDT  measure  of  sensitivity,  d'  adjusted. 

With  respect  to  the  analysis  of  the  hit  probability  data,  a  significant  F 
value  for  the  interaction,  F  (2,50)  =  5.90,  p  =  .0050  and  a  significant  F  value  for 
the  main  effect  of  lateral  localization,  F  (1,25)  =  8.95,  p  =  .0062  were  obtained. 
The  main  effect  for  color  was  not  significant  F  (2,50)  =.62,  p  =  .5422.  Since  a 
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significant  interaction  was  found,  the  Scheffe  procedure  for  pairwise 
comparisons  was  applied.  The  results  of  that  analysis  is  shown  in  table  2. 


TABLE  2 

Summary  of  pair-wise  comparisons  for  hit  probabilities, 
error  rates  and  d'  adjusted 


HIT  PROBABILITY 

ERROR  RATE 

ADJUSTED 

Real /Integrated  >  B  &W/Right 

Real/Right  >Non-reai/Right 

Non-real/Right  >Real/Right 

Real/Integrated  >  Real/Right 

Real /Right  >  Non-real/Inlegrated 

Non-real/Right  >  B  &  W/Right 

Real/Right  >  B  &W/Integrated 

Analysis  of  the  error  rates  produced  a  significant  interaction,  F  (2,50)  = 
5.84,  p  =  0053;  a  significant  main  effect  for  lateral  localization,  F  (1,25)  =8.51), 
p  =  .0074;  and  a  significant  main  effect  for  color,  F  (2,50)  =  3.80  p  =  .0291. 
Results  of  the  pairwise  comparisons  are  shown  in  table  2. 

Analysis  of  the  adjusted  d'  scores  produced  a  significant  interaction, 
F  (2,50)  =  8.18,  p  =  .0008;  a  significant  main  effect  for  lateral  localization,  F 
(1,25)  =4.55,  p  =  .0428;  and  a  significant  main  effect  for  color,  F  (2,50)  =3.38,  p  = 
.0419.  The  results  of  the  Scheffe  procedure  are  summarized  in  table  2. 

Since,  in  all  cases,  the  interactions  were  significant,  no  further  analyses 
of  the  main  effects  data  were  warranted.  Each  of  the  interactions  is  illustrated 
respectively  in  figures  1  through  3. 

DISCUSSION 

Examination  of  the  hit  probabilities  indicate  that  under  the  right 
localized  treatment,  both  the  realistic  and  black  &  white  treatments  scored 
significantly  lower  than  did  the  non-realistic  color  treatment.  Additionally, 
the  non-realistic  group  did  not  differ  significantly  from  ai  y  of  the  treatment 
groups  under  the  integrated  treatment.  This  finding  would  suggest  that  the 
masking  task  inhibited  recognition  under  the  realistic  and  black  &  white 
treatments,  but  did  not  do  so  under  the  non-realistic  treatment.  An 
examination  of  the  error  rates  confirms  tl-.is,  since  both  the  realistic  and  black 
&  white  error  rates  are  high,  with  the  realistic  color  rate  significantly  so.  It  is 
because  of  this  relationship  bet  Areen  hit  rates  and  error  rates  that  the  measure 
of  recognition  sensitivity,  d'  was  derived.  The  statistic,  d'  accounts  for  both 
hit  and  false  alarm  rates  and  is  therefore  a  more  precise  index  of  the  true 
recognition  rate. 

Analysis  of  the  d'  scores  also  indicate  that  the  non-realistic  color  group 
achieved  significantly  higher  than  did  either  of  the  other  two  color 
treatments  under  the  right  localized  treatment.  Additionally,  the  non- 
realistic  color  /right  localized  group  did  not  differ  significantiy  from  any  of 
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the  integrated  treatments,  although  the  mean  d'  value  for  the  non- 
realistic/right  localized  group  was  somewhat  higher 
(see  figure  3). 

Reasons  for  these  findings  can  be  drawn  from  the  literature  on  cerebral 
asymmetry  as  well  as  from  research  on  memory  and  information  processing. 
The  left  cerebral  hemisphere  is  generally  attributed  with  processing  verbal 
and  linguistic  information,  while  the  right  hemisphere  tends  to  specialize  in 
visuospatial  processing.  In  tlie  case  of  the  images  presented  in  this  study,  it 
could  be  concluded  that  the  recognition  process  incorporates  both  types  of 
processing,  as  research  generally  suggests.  The  right  half  processes  the 
visuospatial  component  which  includes  color  information,  while  the  left 
hemisphere  processes  verbal  or  labeling  information  i.e.  tree,  lake,  rocks  etc. 
It  is  this  combination  which  constitutes  "realism",  the  visual  image  plus  the 
verbal  labels  describing  previously  encountered  concepts.  In  the  case  of  the 
right  localized  groups,  the  verbal  label  component  was  inhibited  by  the  verbal 
masking  task,  reducing  the  effectiveness  of  that  form  of  processing.  In  the 
case  of  the  non-realistic  tireatment,  the  additional  color  cues,  which  did  not 
have  any  verbal  or  "realistic"  associations  were  used  purely  as  unique  cues  for 
storage  and  retrieval  of  the  information.  Since  there  were  no  realistic  verbal 
cues,  these  were  not  used  in  the  processing  of  the  images.  In  the  case  of  the 
integrated  treatments,  the  findings  of  previous  studies  (Berry;  1977,  1982, 
1983,1984)  were  upheld,  in  that  realistic  color  materials  are  generally  found  to 
be  superior  to  other  color  formats. 

These  conclusions  are  supported  by  the  now  classic  work  of  Craik  and 
Lockhart  (1972)  and  Craik  and  Tulving  (1975)  which  contends  that  more 
meaningful  material  is  processed  more  deeply.  In  the  case  of  visual  material, 
the  combination  of  imagery  (color  and  sti"ucture)  and  semantic  description 
(verbal  labels)  contiribute  to  more  complete  or  deeper  processing.  When  part 
of  this  information  is  inhibited,  the  remaining  information  is  processed  less 
deeply.  The  lade  of  labels  in  the  non-real  group  is  replaced  by  more  unique 
color  codes  which  serve  to  process  the  information  more  deeply. 

Research  related  to  cerebral  asymmetry  also  supports  this  fmding. 
Beauvois  and  Saillant  (1985)  suggest  that  color  information  is  stored 
separately,  in  both  hemispheres.  This  would  imply  that  if  the  left 
hemisphere  processing  of  color  information  is  inhibited,  the  processing 
would  continue  in  the  right.  Research  by  Jorgenson,  Davis,  Opella  and 
Angerstein  (1980,  1981)  also  suggest  left  hemisphere  color/verbal  processing 
and  right  hemisphere  color  processing.  This  study  inhibited  the  left 
hemisphere  processing  but  permitted  right  hemisphere  color  processing  to 
proceed.  Realistic  color,  which  is  generally  considered  "color"  in  most 
research  is  the  type  of  color  information  processed  in  the  left  half,  since 
realism  implies  semantic  or  verbal  labels.  Right  hemisphere  color  processing 
is  non-verbal  and  thereby  represents  the  type  of  processing  done  with  non- 
realistic  color 

The  primary  conclusion  of  this  study,  therefore,  is  that  color  processmg 
is  bi-locational,  with  realistic/verbal  image  processing  being  done  in  the  left 
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hemisphere  and  pure  color  processing  being  done  in  the  right  hemisphere. 
In  terms  of  color  realism,  it  can  be  concluded  that  visual  realism  probably 

V.WllOmU.kCO    C£    V,V/lliL/iliU.ilV/lL    V/A    11  LAV/A  AAtU.)LlV/A  1/    tJS^tAt.^    VV-AaI^    a4a  ^Ca^Aa  v  a  <^Waa%%. 

being  verbal  in  nature. 

A  v^ord  of  caution  should  however  be  attached  to  the  interpretation  of 
these  findings.  This  study  utilized  a  verbal  (aritlimedc)  masking  task  to 
occupy  and  therefore  inhibit  left  hemisphere  processing  in  the  right  localized 
treatment  group.  It  cannot  be  completely  assured  that  this  task  completely 
engaged  the  processing  of  the  left  half,  but  rathe/  only  interfered  v^th  such 
processing.  It  is  the  suggestion  of  this  researcher  that  further  research  address 
these  questions  using  perceptual  rather  than  verbal  means  of  localizing 
processing  to  either  half  of  the  brain  to  assure  more  complete  localization  of 
processing.  Such  presentation  formats  v^ould  present  different  stimulus 
materials,  simultaneously  to  each  individual  visual  field  (left,  right). 
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Abstract 


Lenin's  1918  decree  on  "Separation  of  Church  ^nd  State", 
updp.ted  in  1975,   reaffirms  atheism  as  the  only  officia] 
stance  for  Christians  in  the  USSR.     This  is  in  spite  of 
the  1000-1500  year  histories  of  some  of  the  established 
churches,  and  the  registration  as  believers  (Christians) 
of  at  least  a  fourth  of  its  people.  DISCRETION  VS.  VALOR 
gives  players  a  chance  to:     identify  with  "believers"  in 
the  USSR  in  order  to  get  new  images  of  these  persons; 
gain  sympathethic  understanding  (empathy)   for  Christians 
in  the  dilemmas  posed  as  tney  exercise  their  faith;  be 
active  participants  in  decision-making  in  a  family  group 
context.     The  24  dilemmas  requiring  choices  from  players 
come  from  research  in  the  USSR,   from  stories  of  recent 
emigres  and  frequent  USSR  travelers,  and  from  American 
and  British  institutes  which  monitor  the  many  legaj.  and 
extra-legal  discriminatory  practices  faced  by  believers. 
The  game  prototype  was  tested,   es^alu^^tod  and  revised 
during  ten  field  tests  in  1987.   During  1988  celebrations 
of  the  millennial  year  of  Russian  Orthodoxy,   72  North 
American  Christians  played  the  revised  game  in  fifteen 
"family"  groups,   returning  self-report  evaluations  which 
included  78  claims  of  "identification"  with  at  least  one 
of  the  game's  characters.  Reports  of  the  game's  greater- 
than-average  influence  upon  their  cognitive  learning 
("5"  or  above  on  a  7-point  scale)  came  from  16%  of  the 
players,  while  60^  of  all  players  reported  that  the  game 
experience  produced  changes  in  their  attitudinal  affect. 
However,   comparison  of  pre-and  post-game  measures  of 
behavioral   intention  for  six  common  religious  practices 
showed  95%  of  players  with  altered  "commitment  levels" 
after  p\ay.     Future  tests  of  "identification  theory"  ana 
"role  empathy"  are  suggested  by  this  study. 
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Introduction 


In  1990,   it  is  difficult  for  Americans  to  remember 
the  "bad  old  days  of  the  cold  war"  with  the  rapid  changes 
that  have  taken  place  in  Eastern  Europe  during  the  past 
year*     Mass  demonstrations,  calls  for  independence  within 
the  Soviet  Union  itself,   free  exercise  of  election  rights 
and  the  promise  of  multiple  party  elections  from  Soviet 
President  Gorbachev  testify  to  glasnost  and  perestroika 
in  the  political  arena .     There  are  even  some  signs  of 
increased  religious  openness  in  tnis  offically  atheistic 
nation  although  church  leaders  are  cautious  in  their  pro- 
nouncements.    London's  Keston  College  and  the  American- 
based  Institute  for  Religion  and  Democracy,  watchdogs  on 
all  matters  of  rel igious  freedom,   report  release  of  large 
numbers  of  previously  held  "prisoners  of  conscience'*.  Thf? 
return  to  the  parishes  of  several  hundred  "non-working" 
Russian  Orthodox  churches,  previously  designated  museums 
or  public  buildings,   seems  to  demonstrate  acknowledgment 
by  the  Councils  on  Religious  Affairs  that  insistence  on 
atheism  as  the  only  official  beaief  has  not  succeeded  in 
stamping  out  the  latent  faith  of  Soviet  "believers"  (the 
name  used  for  all  Christian  groups  in  the  USSR),  January 
8  media  coverage  of  thousands  of  Russian  Orthodox  parish 
members  openly  celebrating  Christmas,  many  for  the  first 
time  since  Lenin  *  s  1918  decrees ,  gave  credibi lity  to  a 
possible  new  era  in  church-state  relations.  Even  the  long 
fight  by  Ukranian  Catholics  for  rsccgnition  as  a  separate 
denomination  is  given  some  hope  for  dialogue  by  Gorbachev 
(February  1,  New  York  Times).  Actual  legal  enactments  are 
still  lacking,  so  it  remains  to  be  seen  if  both  legal  and 
extra-legal  long-standing  societal  discr irainat ion  will 
continue  to  be  exerted  against  believers.  Still,  American 
Christians  may  see  some  hopeful  signs  for  Soviet  brethren 

This  was  not  the  case  in  1986  when  the  present  study 
was  begun.     Americans  still  had  very  limited  information, 
and  hence  a  curtailed  view  of  persons  and  conditions  m 
countries  behind  the  "Iron  Curtain".     When  the  Fellowship 
on  Reconciliation  undertook  to  promote  more  understanding 
about  ordinary  Russian  people,   they  entitled  their  photc 
series,   "Forbidden  Faces.'     Media  coverage  about  persons 
An  the  Soviet  Union,   rather  than  helping  Americans  to 
form  sympathetic  or  even  accurate  images,  added  to  their 
misconceptions,  when  it  pictured  Russians  as  unfriendly 
enemies.     The  pre-summit  characterization  of   che  Soviet 
"evil  empire"  by  the  American  President  did  little  to 
alleviate  natural  antipathy  caused  by  lack  of  information 
or  even  intentional  build-up  of  distrust  by  the  media. 

Misconceptions  were  even  stronger  for  American  church 
members  with  the  specter  of  atheism  added  to  the  overall 
negative  attitudes.  Soviet  publications  usually  portrayed 
religious  co.nmunities  as  being  enfeebled  and  dominated  by 
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"old,   superstitious  babushki",   suggesting  the  death  of 
the  church  when  these  Russian  grandmothers  w^re  gone. 
The  more  frequent  criticism  was  that  Russian  Christians 
had  so  compromised  the  fa:th  that  it's  authenticity  was 
gone.     Both  American  and  Canadian  press  releases  were 
critical  of  Soviet  church  representatives  at  Vancouver's 
1983  World  Council  of  Churches  Assembly  for  neglecting  to 
use  the  occasion  to  chastise  the*r  government's  position 
on  religion  and  human  rights. 

The  former  discrimination  faced  by  Soviet  Jewry  and 
refusniks  was  harshest  of  all,  but  this  svibject  received 
much  more  media  attention  and  public  outrage  on  this* 
ccn'»~inenc.     By  contrast,  North  American  Christians,  even 
those  who  might  have  been  disposed  to  be  sympathetic  to 
their  USSR  counterparts,  had  little  information  about 
those  persons  v^hom  the  Soviet  state  branded  as  believers. 
Vague  knowledge  about  Lenin's  post-revolution  decrees 
about  atheism  as  the  only  officially  endorsed  position 
produced  more  prejudice  than  sympathetic  understanding. 

DISCRETION  VS.   VALOR  was  developed  to  address  this 
need  by  supplementing  other  publications  produced  by  the 
National  Council  of  Churches.'   Friendship  Press  for  its 
1988  study  of  Peojcvles  and  Churches  of  the  Soviet  Union. 
In  this  year  of  the  millennium  of  the  Russian  Orthodox 
Church,  a  stated  goal  called  for  a  change  in  attitudes. 

...   to  bring  about  a  recognition  of  our  prejudices, 
fears  and  distorted  images,   so  that  we  (North 
American  Christians)  may  be  open  to  fresh 
insights  about  people  and  churches  in  the  USSR. 

A  goal  of  "being  open  to  fresh  insights"  suggested 
that  some  learning  objectives  for  the  1988  Soviet  study 
would  involve  attitudinal  change.     Designers  and  users  of 
simulation  games  claim  that  this  more  experiential 
learning  r-^thcd  makes  gaming  particularly  effective  in 
producing  changes  in  attitudes.     Four  advantages  claimed 
for  simulation  gaming  include:  potential  for  attitudinal 
change;   the  realism  that  comes  from  group  interaction  in 
decision-making;     possible  "identification"  with  persons 
whose  roles  are  played  (Williams,   1986)   and  role  empathy 
(Livingston  et.  al..  The  Hopkins  Games  Program,   1973)  or 
sympthetic  understanding  about  those  who  are  portrayed. 

The  purpose  of  this  study  was  to  develop  and 
evaluate  a  simulation  game  involving  participants' 
identification  with  and  sympathetic  understanding  of 
those  persons  who  are  called  "believers"   (Christians)  in 
the  Soviet  Union.     Three  learning  objectives  would  guide 
the  formative  evaluation  process;  criterion-referenced 
summative  evaluation  would  be  conducted  after  revisions. 
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Methods  of  development  and  formative  evaluation 


In  1986,  the  author  composed  seven  character  roles 
and  28  decisions,  called  "commitment  level"  in  the  game, 
which  typified  Soviet  believers*  actions.     In  spring  of 
1S87,  a  research  grant  provided  opportunities  to  travel 
for  several  weeks  to  the  USSR  to  interview  believers  in 
a  variety  of  settings*     Following  these  interviews,  six 
characters  were  retained,  roles  were  revi^^ed,  playing 
rules  were  developed,  and  sets  of  cards  were  constructed 
to  portray  four  levels  of  COMMITMENT  and  RISK  chat  each 
of  the  Soviet  characters  might  experience  under  present 
discriminatory  regulations,  both  legal  and  extra--legal  • 

The  research  trip  to  the  Soviet  Union  also  afforded 
opportunities  to  secure  visuals  to  supplement  the  game* 
A  42-slide  set.  Picture  Yourself ,  A  Soviet  Believer^ 
accompanies  the  game  and  acquaints  the  players  with  each 
other *s  decision  choices. 

Content  review  of  game  rules,   cards,  and  roles  by 
subject  matter  experts  was  done  by  five  frequent  Soviet 
travelers,   defined  as  those  whose  work  within  agencies  in 
the  USA  caused  them  to  travel  repeatedly  to  the  USSR  and 
to  have  extensive  contacts  with  Soviet  Christians. 
Included  were  Pittsburgh's  Cultural  Affairs  Chairman  of 
Peace  Links,  the  head  of  the  Chicago-based  Institute  for 
the  Study  of  Marxism  and  Christianity,  and  the  lone  U.S. 
Protestant  representative  from  the  National  Council  of 
Churches  to  the  USSR,   in  residence  there  from  1983-1986. 
The  initial  design  review  was  done  by  Bailey  and  Tarasar 
(1987),   the  authors  of  the  Friendship  Press  guidebook  for 
this  ecumenical  study. 

Throughout  June  and  July  of  1987,  Field  Trials  of  the 
game  were  conducted  with  populations  of  prospective 
teachers  who  were  preparing  to  provide  leadership  for  the 
1988  millennial  study.     Two  university  classes,  one  in 
Games  and  Simulations  and  another  in  Inter cultural 
Communication,  participated  in  Field  Trails  of  the  game 
and  provided  suggestions  for  eight  revisions  of  the  game. 
A  total  of  eleven  Field  Trails  were  conducted,  involving 
228  players  on  campuses  in  four  eastern  states. 
(Figure  l:   Flowchart  of  Phase  2  &  3  Evaluation  Processes) 

Use  of  several  types  of  assessment  forms  provided 
bases  for  evaluation  of  these  instruments  during  the 
field  tests.     Two  specialists,  one  in  evaluation 
and  one  in  test  design,  provided  expert  review  of  the 
instruments  judged  to  have  the  greatest  usefulness  in 
formative  evaluation.     (See  current  forms  in  Appendix.) 
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Summat Ive  evaluation  of  simulation  game ' s  objectives 


By  1988,  the  year  of  the  Russian  Orthodox  Church's 
millennium  and  the  year  of  the  ecumenical  study  by  North 
American  Christians,  the  game  was  ready  for  use  in  eight 
additional  Field  Tests.  In  five  settings  the  75  players 
Vi/ere  members  of  Christian  denominations  and  data  from  the 
revised  evaluation  instruments  were  analyzed  in  summative 
evaluation  about  the  game's  effectiveness  in  meeting  its 
criterion-referenced  obi"  ::tives. 

The  current  Decision  Predictor  (Appendix  A)  is 
designed  to  demonstrate  changes  in  sympathetic  under- 
standing for  Soviet  Christians  who  are  faced  with  daily 
dilemmas  as  they  practice  their  faith.     The  participant 
is  asked  to  respond  as-if  s/he  were  seeking  to  exercise 
some  religious  practice  in  the  Sovet  Union.  Attitude 
change  is  measured  on  an  agreement  scale  of  [+3  to  -3], 
a  shift  in  the  direction  or  strength  of  a  self-report  of 
one's  own  predicted  be*iavior.     Comparisons  of  responses 
on  this  Decision  Predictor,   both  before  and  after  play, 
are  also  useful  in  producing  discussion  during  the  game 
debriefing  time. 

The  Revised  Evaluation  Form  (Appendix  B)  elicits 
information  to  measure  the  game's  criterion-referenced 
objectives.     It  also  serves  as  a  means  of  continued 
evaluation  and  revision  of  the  game.     Some  unexpected 
patterns  in  results  frcm  earlier  evaluations  indicated 
that  participants  might  be  experiencing  role  empathy  or 
identification  with  the  Soviet  citizens  whom  they  play  in 
the  simulation.     "Identification"  is  measured  by  a  self 
report,   the  ability  of  a  person  to  see  similarities 
between  himself  and  another.     "Role  empathy"  is  defined 
as  the  process  of  "feeling  into"  another  person  with 
"sympathetic  (affective  domain)  understanding  (cognitive 
elements) . 

Table  1  shows  the  data  which  measure  results  for 
Objective  1  from  the  game:   "to  cause  identification  with 
Soviet  believers"  by  North  American  Christian  players. 
Responses  from  76  players  yielded  78  reports  of  identi- 
fication with  one  or  more  characters  in  the  game.  All 
but  ten  players  reported  identification  with  at  least 
one  character.     Table  1  also  shows  a  response  pattern  in 
these  reports  of  identification:   the  greatest  percentage 
of  claims  came  from  players  who  played  that  specific  role 
during  the  game,  supporting  Williams*  identification 
theory  (1986).     Game  players'   reports  of  identification 
met  the  75%  criterion  for  game  Objective  1. 
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Tatle  1 


Response  by  Jo  North  American  Christians  to  Question  9 
"Check  any  character  with  whom  you  could  identify, " 

#,  %  of  Characters  Claimed  in  Identification 
Father    Mother     Daughter    Son  G, mother  Priest 

Character  N 


Tables  2  and  3  show  analysis  of  data  in  assessing 
how  well  Objective  2  for  the  game  i-^as  met:    "to  i'lcrease 
sympathetic  understanding  about  Soviet  believers  who 
experience  dilemmas  in  the  practice  of  their  faith". 

Two  of  the  self-report  questions  on  the  Evaluation 
Form  (Appendix  B)  provide  data  in  assessing  affective  as 
wel]  as  cognitive  reactions  after  playing  the  game. 
Question  5  asks  about  the  usefulness  of  the  simulation 
experience  in  learning  nev/  information  with  76%  of  the 
players  reporting  a  rating  of  "5"  or  above  on  a  7-point 
scale.     Only  60%  of  players  report  such  high  ratings  on 
Question  6  about   che  game's  usefulness  in  attitude 
change,   falling  short  of  the  75%  criterion. 

However,  when  the  raw  data  from  the  Decision 
Predictor  are  examined,  all  but  four  players  reported 
some  change  in  their  predicted  behavioral  response  to  at 
least  one  of  the  six  dilemmas  presented  during  the  game. 
For  purposes  of  this  study,   there  is  no  right  or  wrong 
answer,   and  a  change  in  either  direction  can  be  counted 
as  a  sign  of  some  shift  in  attitude  after  play.     Table  2 
shows  such  shifts  for  20  players  at  Test  Site  #15.  Table 
3  then  summarizes  data  from  all  five  test  sites. 
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Table  2 

Site  #15.   Extent  of  each  player ' s  change  on  six  dilemmas 
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Table  3 

Average  change  for  all  players  using  Decision  Predictor 


Field  Tests,     5  i\ .  A.   Christian  groups 


Category 

#14 
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#18 

Total 

Question  #1 
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17 

4 

123 

Question  #6 

15 

27 

10 

7 

6 

65 

Total  spaces  shifted 

143 

159 

91 

110 

76 

579 

Total  #  of  players 

17 

20 

12 

15 

8 

72 

Average  change 

8.41 

7  .  95 

7  .  50 

7  .  33 

9.5 

8.  04 
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The  extent  of  shift  on  post-game  questionnaires  of 
intended  behavioral  response  shows  an  average  change 
on  this   [-*-3  to  -3]   empathy  scale  that  varies  from  7.33 
spaces  for  Test  Site  17  to  9.5  spaces  shift  for  Site  18. 
So  far  then,   summative  evaluation  suggests  that  the  game 
provides  greater  cognitive  information  than  attitudinal 
shifts  on  players '   sel f  reports .     However ,   responses  of 
intended  behavior  in  either  direction  on  empathy  scales 
show  changes  in  players*   ratings  and  support  for  the 
game's  objective  of  greater  sympathetic  understanding  for 
Soviet  believers  and  the  dilemmas  which  they  face. 

The  third  objective  of  the  game  is  "to  create  a 
setting  in  which  the  influence  of  the  group  can  be 
brought  to  bear  upon  decision  making."     This  objective  is 
measured  by  Questions  7  and  8  on  the  Evalution  Form. 
Only  one  third  of  the  players  report  that  the  influence 
on  them  from  other  players  was  "stronger  than  average". 
On  the  second  question,   only  28%  of  the  players  report 
"greater  than  average"  success  in  influencirg  others* 
decisions  during  the  game.     As  presently  us^d  the  game 
does  not  meet  the  predicted  criterion  level  for  influence 
from  the  group  discussion  context. 


Use  of  the  Decision  Predictor  and  Evaluation  Form 
proved  useful  in  formative  evaluation  of  the  simulation 
game,   DISCRETION  VS.  VALOR,   through  eight  1987  revisions 
These  measures  also  proved  valuable  during  1988  field 
tests  by  75  North  American  Christians  playing  the  revised 
DISCRETION  VS.   VALOR  in  five  different  settings.  Data 
were  gathered  about  how  well  the  simulation  game  met  its 
objectives  and  how  well  the  criterion-referenced  research 
Hypotheses  were  supported.     Results  support  Williams^ 
"identification"  theory  and  suggest  further  study.  Use 
of  a  empathy  scale  provided  data  on  behavioral  intention 
to  supplement  players*   self-reports  of  affective  changes. 
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REVISED  APPENDIX 


DISCRETION  VS.  VALOUHt  A  Simulation  by  Barbara  BlacKetone 

DECISION  PREDICTOR   ^Before  playing  the  garner   After  playing 

Pretend  that  you  have  to  make  each  of  these  decisions  in  the  USSR. 
Circle  the  one  that  best  decribes  what  you  would  do  in  this  situation. 

Definitely      Probably       Maybe  Maybe         Probatly  Definitely 

Yes  +3         Yes  +2         Yes  +1         Not  -1         Not  -2  Not  -3 

1.  If  I  had  a  Bible  and  access  to  some  study  guides «  I  would  invite 
other  Baptists  to  my  home  to  read/study  the  Bible  together.  •f3"»-2-t- 1-1-2-3 

2.  If  I  had  some  musical  ability,  I  would  give  practice  in  learning 
hymns  to  my  son  and  two  of  his  Baptist  friends  in  our  home.  +3+2+1-1-2-3 

3.  If  I  were  a  17-year-old  in  need  of  Soviet  university  education,  I'd 
give  up  open  worship  &  try  to  be  a  good  Komsomol  member.  +3+2+1-1-2-3 

4.  If  I  were  an  11-year-old  with  athletic  aspirations,  I'd  join  the 
Young  Pioneers  even  it  meant  that  I  couldn't  wear  my  cross.  +3+2+1-1-2-3 

5.  As  a  grandmother  on  a  pension  in  Moscow,  I'd  probably  write  letters  to 
the  newspapers  if  I  felt  human  rights  were  being  violated.  +3+2+1-1-2-3 

6.  If  my  Orthodox  priest  friend  were  in  prison  for  his  outspokenness,  I 
would  visit  him  and  attempt  to  smuggle  in  a  Bible  to  him.      +3h -.+1-1-2-3 


DECISION  PREDICTOR   ^Before  playing  the  game;   After  playing 

Pretend  that  you  have  to  make  each  of  these  decisions  in  ♦he  USSR. 
Circle  the  one  that  best  decribes  what  you  would  do  in  this  situation. 

Definitely      Probably       Maybe  Maybe         Probably  Definitely 

Yes  +3         Yes  +2         Yes  +1         Not  -1         Not  -2  Not  -3 

1.  If  I  had  a  Bible  and  access  to  some  study  guides,  I  would  invite 
other  Baptists  to  my  home  to  read/study  the  Bible  together.  +3+2+1-1-2-3 

2.  If  I  had  Pome  musical  ability,  I  would  give  practice  in  learning 
hymns  to  my  son  and  two  of  his  Baptist  friends  in  our  home.  +3+2+1-1-2-3 

3.  If  I  were  a  17-year-old  in  need  of  Soviet  university  education,  I'd 
give  up  open  worship  &  try  to  be  a  good  Komsomol  member.  +3+2+1-1-2-3 

4.  If  I  were  an  11-year-old  with  athletic  aspirations,  I'd  join  the 
Young  Pioneers  even  it  meant  that  I  couldn't  wear  my  cross.  +3+^^+1-1-2-3 

5.  As  a  grandmother  on  a  pension  in  Moscow,  I'd  probably  write  letters  to 
the  newspapers  if  I  felt  human  rights  were  being  violated.  +3+2+1-1-2-3 

6.  If  my  Orthodox  priest  friend  were  in  prison  for  his  outspokenness,  I 
would  visit  him  and  attempt  to  smuggle  in  a  Bible  to  him.  +3+2+1-1-2-3 

Circle  the  role  you  played:  father-mother-daughter-son-grandmother-priest 
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RHVT^Fn-ADDPwnTY  O 
GAME  EVALUATION  FORM  for  DISCRETICS  Va.  VALORi 
Pl'^ae  circle  one  number  and  return  to  B.i-bara  Blackstone.  Date.  

1.  How  much  did  you  enjoy  playing  the  game? 

Very  little    '  ^  2^  ~  ^      ..  ^  . 

Very  much 

2.  HOW  efiay  was  it  to  follow  the  in.tructione  and  rules  of  the  game? 

*  Very  difficult'  '  ~  ^  —L  7 


V«*ry  eaoy 

3.  HOW  comfortable  were  you  with  the  length  of  time  that  it  took  to  play? 

1  2   3  4  c  ^ 

very  uncomiortable  "   —  2.  ■■    ^        ^    .  7, 

very  comtortable 

4.  IF  you  were  uncomfortable,  wa«  it  too  short?   or  too  long? 

fbSu't^S^^it'SeSvrrr  ^^""^"^  informati~ 

1   2   3  4c 

Tlo^Kelp  received  2  ,  ^         .       T  , 

very  helpful 

I^utioeferbeJuve'ri;  "^"^'^"^  ^  ^^^^'^^  Vour  opinion. 

NO  Change  ^  =  2  ^   6  7 

A  great  deal 

'dufiS:  ^jr^ouJiJ  °of  -'^^  -iser  decision. 

L  influence  "  ~  ^  5  ^  7 

Strong  influence 


1  ,  2 
NO  mtiuence 


Very  succeesful 
55uSe;?^?m::{;:-LS£r-M)f^^ 

ftther?_    mothe.?_     .on?_     daughter?^  prie.t?_  grandmother?^ 
Then  circle  the  character  that  you  played  during  the  game. 


ERIC 


033 
95 


Title 


A  Model  for  the  Motivational  Instruction  of  Adults 


Authors: 

Roy  M.  Bohlin 
William  D.  Milheim 
Karen  J.  Viechnicki 


085 

96 


A  Model  for  the  Motivational  Instruction  of  Adults 


This  paper  provides  educators  with  a  model  for  the 
instructional  motivation  of  adults.  Adult  learning  theory  has 
been  integrated  into  John  Keller's  instructional  motivation 
model  to  develop  motivational  needs-assessment  survey 
instruments.  Results  of  research  eliciting  adults'  perceptions  of 
their  motivational  needs  were  analyzed.  Results  show  that:  (a) 
instructional  motivation  has  components  of  instructional 
appeal  and  learner  effort  and  (b)  specific  instructional 
strategies  are  important  for  the  motivation  of  adult  learners, 
A  model  for  the  instructional  motivation  of  adult  learners  is 
presented.  This  model  provides  an  important  step  in  the 
integration  of  adult  learning  theory  into  the  design  of 
motivational  instruction  for  adult  learners. 

Introduction 

Demographics  suggest  that  during  the  next  several  years,  the  primary  growth 
areas  in  U.  S.  education  will  be  in  adult  and  continuing  education.  By  1992,  half 
of  all  college  students  will  be  over  25  and  25%  will  be  over  35  (Hodgkinson, 
1985).  In  addition,  our  American  culture  encourages  adults  to  enroll  in  schooling. 
Recently,  a  group  of  experts  presented  a  report  to  President  George  Bush  in  which 
they  stated  that  Americans  will  increasingly  change  not  just  their  jobs  but  their 
occupations,  and  will  need  regular  retraining  to  learn  new  skills  -  a  life  time  of 
learning  (American  Agenda,  1988).  This  need  for  continuing  education  is 
increasingly  being  addressed  by  institutions  of  higher  education,  business  and 
industry,  and  community  agencies.  We  need  to  identify  instructional  strategies  that 
best  facilitate  the  continuing  motivation  of  adult  learners. 

The  purpose  of  this  paper  is  to  provide  educators  with  a  model  for  the 
motivating  instruction  of  adult  learners.  This  has  been  accomplished  by  linking 
significant  adult  learning  theory  with  motivational  instructional  design;  adapting 
survey  needs  assessment  instruments  from  Keller  and  Subhiyah  (1987);  administering 
these  instruments  to  adult  learners;  analy/dng  the  needs-assessment  responses;  and 
outlining  a  model  for  the  instructional  motivation  of  adult  learners. 

Theoretical  Framework 

The  underlying  theory  base  for  instruction  of  adults  comes  from  two  diverse 
arenas  -  adult  learning  theory,  and  instructional  design  for  motivation.  John  Keller 
(1983)  supplies  a  framework  for  the  design  of  motivational  instruction  for  learners 
of  all  ages.  Keller's  ARCS  model  is  based  upon  inductive  analyses  of  the  actual 
teaching  practices  of  highly  motivating  instructors  and  deductive  analyses  of  current 
learning  and  motivation  theories  (such  as  the  work  of  such  theorists  as  Gagne, 
Bruner^  Bandura,  Weiner,  and  Malone).    Keller's  ARCS  model  integrates  a  great 
amount  of  what  we  know  about  motivating  learners,  and  supplies  an  organization  for 
theory  and  practical  applications. 
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Motivational  Instructional  Design.  Instructional  motivation  attracts  learners 
toward  the  instruction  and  increases  their  effort  in  relation  to  the  subject  (Keller, 
1983).  This  means  that  instructional  motivation  has  two  components,  appeal  and 
effort  generation.  Motivational  instruction,  therefore,  has  appeal  for  the  learner  and 
stimulates  learner  effort. 

Keller  and  Suzuki  (1988)  and  Keller  and  Kopp  (1987)  also  identify  four 
categories  of  motivational  "conditions"  in  learning  situations:  attention,  relevance, 
confidence,  and  satisfaction.  For  facilitation  of  continuing  motivation,  these  four 
conditions  or  stages  should  be  addressed  Although  these  "conditions"  (such  as 
expectancy  for  success)  interact  with  instructional  methods  to  cause  learner 
behaviors  or  attitudes,  the  "conditions"  cannot  be  manipulated  directiy  by  the 
instructor.  It  is,  instead,  the  methods  or  strategies  of  instruction  that  are  under  the 
instructor's  control  which  produce  various  instructional  consequences.  The 
instructor,  therefore,  promotes  motivation  by  using  appropriate  strategies. 

The  consequences,  which  result  from  the  interaction  of  the  methods  with  the 
learner's  conditions,  arc  termed  "outcomes."  Keller's  model,  therefore,  is  designed 
to  provide  effective  instructional  "methods"  under  particular  "conditions"  to  yield 
desirable  motivational  "outcomes."  The  model,  therefore,  contains  specific  methods 
or  strategies,  that  are  aimed  at  producing  motivational  outcomes,  when  learners  are 
lacking  sufficient  conditions,  such  as  interest  or  motives.  The  initials  of  these  four 
categories  -  attention,  relevance,  confidence,  and  satisfaction-give  Keller's  model 
tlie  acronym  ARCS. 

The  first  requirement  for  modvatimg  instruction  is  to  gain  and  maintain  the 
attention  of  the  learner.  This  can  be  achieved  through  several  procedures  aimed  at 
increasing  his/her  curiosity  or  arousal,  through  humor,  paradoxes,  inquiry,  etc. 

Once  attention  is  aroused,  the  learner  evaluates  the  relevance  of  the 
environment  before  becoming  highly  motivated  The  learner  must  also  perceive  that 
significant  personal  needs  are  being  met  by  the  instruction.  This  can  be  facilitated 
by  matching  instruction  to  learners'  goals,  using  metaphors,  cooperative  atmosphere, 
etc. 

Confidence  is  related  to  the  learner's  attimde  toward  success  or  failure.  This 
attitude  influences  his/her  actual  effort  and  performance.  Confidence  can  be 
cultivated  by  cleari;'  indicating  the  requirements  for  success,  allcA^ing  learner 
control,  using  leiim\ig  organizers,  etc. 

Lastly,  indivi  iual  satisfaction  is  important  for  sustaining  motivation.  Learners 
must  perceive  the  *ewards  gained  as  appropriate  and  consistent  with  their 
expectations.  Leainer  satisfaction  can  be  addressed  through  providing  appropriate 
recognition  for  success,  giving  informative  and  corrective  feedback,  supporting 
intrinsic  motivation,  etc. 

It  is  expected  that  the  motivational  factor  of  instructional  appeal  (how 
interesting  and  attractive  the  learner  views  the  instruction)  is  more  closely  related  to 
the  categories  of  attention  and  relevance.  The  motivational  factor  of  learner  effort 
(the  amount  of  effort  a  learner  decides  to  give  toward  learning)  is  more  closely 
related  to  tiie  categories  of  confidence.  Learner  satisfaction  is  expected  to  be 
related  to  both  interest  and  effort.  These  predicted  relationships  were  investigated  in 
this  study. 
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Adult  Learning  Theory.  Literature  in  adult  education  cites  many  strategies 
which  are  generalizable  across  levels  of  instruction.  Authors  such  as  Knowles 
(1980),  Cross  (1981),  and  Zemke  and  Zemke  (1981),  however,  feel  that  adult 
learners  have  different  instructional  needs  than  younger  students.  These  authors 
suggest  strategies  they  feel  to  be  particularly  important  for  the  instructional 
motivation  of  adults. 

In  order  for  adults  to  learn,  they  must  be  interested  in  an  issue  or  subject,  and 
it  must  grab  their  attention  (Knowles,  1980).  Adult  learners  usually  arc  prepared  to 
take  responsibility  for  making  choices  about  their  own  learning.  Each  adult  student 
needs  to  be  able  to  negotiate  an  individual  course  of  study  tliat  is  relevant  to  his  or 
her  own  needs  (Hull,  1981).  Adults  seek  out  learning  because  they  have  use  for  the 
knowledge  or  skill  being  taught  (Zemke  &  Zemke,  1981).  Learning  is  a  means  to 
an  end,  not  an  end  in  itself. 

Adults  must  feel  competent,  exhibit  confidence  during  learning,  and  should 
feel  at  ease  in  the  learning  environment  (Knowles,  1980).  Knowles  states  that 
confidence  is  built  through  self-evaluation,  by  comparing  performance  skills  before 
and  after  the  learning  experience.  This  results  in  the  re-diagnosis  of  learning  needs. 
Further,  expectations  of  the  teacher  arc  merged  with  those  of  the  learner.^  Tough 
(1978)  explains  reasons  adults  choose  their  own  way  of  learning  by  desiring  to:  (1) 
set  their  own  learning  pace,  (2)  use  their  own  style  of  learning,  (3)  keep  the 
learning  flexible  and  easy  to  change,  and  (4)  put  their  own  structurc  on  the  learning 
project.  Zemke  and  Zemke  (1981)  report  that  when  adult  learners  arc  asked  to  risk 
new  behavior  in  front  of  peers  and  cohorts,  their  self-esteem  and  ego  arc  on  the 
line.  They  must  feel  confident  beforc  they  can  perform  in  these  situations. 

According  to  Manteuffel  (1982)  satisfied  learners  arc  described  as  involved, 
challenged,  self-directed,  rcwarded,  and  safe  (i.e.,  feel  comfortable  to  ask  "stupid" 
questions).  Zemke  and  Zemke  (1981)  state  that  an  adult  seeks  to  incrcase  or 
maintain  his/her  sense  of  self-esteem  and  pleasure,  which  rcsults  in  a  feeling  of 
satisfaction. 

It  has  been  suggested  that  some  instructional  strategies  are  more  important  for 
adult  learners  than  for  younger  students.  The  integration  of  adult  learning  theory 
and  the  ARCS  model  provides  a  framework  for  a  potential  adult  motivation  model. 
This  integration  was  analyzed  through  a  needs-assessment  instrument  based  upon  the 
literature  in  both  theory  bases. 

Methodology 

This  study  surveyed  adults  in  a  variety  of  settings  to  determine  tlieir 
perceptions  of  their  motivational  needs.  The  subjects  for  the  present  study  were  307 
adult  participants  in  Continuing  Education  at  a  major  midwestem  university  from  a 
variety  of  different  classes  and  workshops,  both  credit  and  non-credit.  While  the 
majority  of  subjects  were  teachers  and  administrators  of  grades  K-12,  other 
occupations  were  also  represented.  Involvement  in  the  study  was  voluntaiy  and  the 
sample  included  both  men  and  women  whose  average  age  was  approximately  39 
years.  Subjects  were  given  a  survey  instrument  during  a  course  or  workshop  in 
order  to  deterr.iine  their  perceptions  concerning  their  generalized  personal  motivation 
needs. 
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Instruments.  The  needs  assessment  instruments  were  developed  through  a 
revision  of  the  Course  Interest  Sur/ey  (CIS)  by  Keller  and  Subhiyah  (1987). 
Because  the  CIS  was  developed  to  evaluate  a  specific  course,  the  survey  Nvas 
revised  by  the  authors  for  better  application  to  general  instructional  strategies.  This 
was  accomplished  by  a  series  of  refinements  from  an  earl'er  version  of  die 
instrument  (Viechnicki,  Bohlin,  &  Milheim,  1989).  The  statements  were  reworded 
in  two  ways,  they  were  changed:  (a)  from  past  tense  to  present  tense  and  (b)  to 
refer  more  direcdy  to  instructional  methods.  Two  statements  diat  applied 
specifically  to  the  quality  of  materials  were  omitted  Seven  statements  comprised  of 
strategies  suggested  by  the  literature  but  not  included  in  die  CIS  were  dien  added. 
The  Likert-type  choices  were  changed  from  statements  about  actual  instructor  use  to 
statements  about  die  general  effect  of  such  strategies  on  eidier:  (1)  appeal  of  die 
instruction  -  Course  Interest  Survey,  Revised  (CISR);  or  (2)  effort  of  die  learner  - 
Course  Effort  Survey,  Revised  (CESR).  The  resulting  two  instruments  are  42-item 
five  point  Likert-type  scales  assessing  learners  perceptions  of  die  effects  of  given 
instructor  strategies  on  dieir  interest  and  effort  in  instructional  settings  (see  Table  1). 

The  Course  Interest  Survey  Revised  (CISR)  and  die  Course  Effort  Survey 
Revised  (CESR),  dierefore,  include  items  drawn  from  die  integration  of  adult 
learning  and  general  motivational  factors  also  relevant  to  adult  learners.  The 
instruments  are  designed  to  measure  respondents'  perceptions  of  dieir  instructional 
needs  related  to  instructional  appeal  (CISR)  and  learner  effort  (CESR).  Typical 
demographic  questions  have  also  been  added  to  both  instruments. 

Each  of  die  statements  was  rated  by  aU  subjects  on  a  five-point  scale  as  to 
die  expected  effect  widi  die  number  5  indicating  a  very  positive  effect,  and  1  a  very 
negative  effect.  The  test-retest  reliabilities  of  the  instruments  over  a  two-week 
period  were  found  to  be  .69  (CISR)  and  .71  (CESR). 

Procedure.  The  administration  of  die  instrument  was  carried  out  over  a  four 
mondi  period  in  a  number  of  different  classes  and  workshops.  The  classes  and 
workshops  were  selected  to  give  a  balanced  stratified  sampling  diat  reflected  a 
general  graduate  and  continuing  education  population.  After  receiving  instiactions, 
all  subjects  worked  dirough  die  survey  form  and  recorded  their  answers  to  each 
item  on  a  machinc-scorable  answer  sheet  Each  administration  of  diis  survey 
required  approximately  fifteen  minutes  of  class  time. 

Data  analyses  were  carried  out  on  an  IBM  4381  using  SPSS-X.  In  addition  to 
item  frequencies  across  all  subjects  and  subgroups  for  bodi  surveys,  die  analyses 
also  included  t-tests  to  determine  whether  any  numerical  differences  for  items  and 
for  sub  scales  across  die  two  surveys  were  significant  for  diis  group. 

Results 

Most  of  die  items  were  rated  by  die  respondents  as  having  at  least  a  slightiy 
positive  effect  on  dieir  interest  and  effort  (see  Tables  2  and  3).  The  very  high 
ratings  of  many  of  die  items  suggests  a  possible  ceiling  effect.  The  results, 
however,  can  be  used  to  analyze  certain  trends. 

Responses  differed  on  the  two  instruments  in  several  ways.  Ratings  were 
higher  for  interest  dian  for  effort  on  41  of  die  42  items.  T-tests  showed  die  ratings 
on  die  CISR  to  be  significandy  higher  (p  <  .05)  for  18  of  die  items,  and  for  all 
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four  subscales. 

The  42  items  were  also  ranked  by  mean  response  on  each  instrument  (see 
Table  4).  As  expected,  there  were  differences  in  the  rankings  of  many  items  on  the 
two  instruments.  Large  differences  in  rankings  between  the  two  instruments  were 
defined  as  those  ordinal  differences  in  rankings  above  the  third  quartile  (ordinal 
differences  of  6  or  more  for  this  data).  These  large  differences  in  the  rankings 
between  the  instruments  were  also  identified  (see  Table  4)  and  analyzed. 

Using  this  criteria,  four  items  in  the  confidence  subscaie  had  large  differences 
in  rankings  and  all  four  strongly  favored  perceived  effort.  This  seems  to  supports 
the  contention  that  confidence  is  more  closely  related  to  effort  than  to  interest  The 
relevance  and  satisfaction  subscales  each  had  two  items  with  large  differences 
favoring  interest  and  one  item  with  a  large  difference  favoring  effort.  The  attention 
subscaie  showed  no  large  differences  in  rankings. 

Conclusions 

These  results  suggest  th^at  instructional  strategies  can  have  a  positive  effect  on 
the  interest  and  effort  of  adult  learners.  Differences  in  the  mean  responses  and  in 
the  rankings  of  the  items  also  support  the  definition  of  instructional  motivation  as 
including  both  instructional  appeal  and  learner  effort.  The  results  further  suggest 
that  many  instructor's  motivational  methods  are  perceived  to  have  a  stronger 
positive  effect  on  adult  learners'  interest  than  on  adult  learners'  effort  in  learning. 
It  is  easier  to  stimulate  arousal  and  interest  than  to  impact  on  the  effort  of  adult 
students.  Confidence  building  strategies  were  found  to  be  much  more  strongly 
linked  to  perceived  effort  of  learners  than  to  instructional  appeal.  Attention, 
relevance  and  satisfaction  promoting  strategies  were  not  clearly  linked  more  strongly 
to  interest  or  to  effort 

Based  upon  instructional  motivation  theory,  adult  learning  theory,  and  these 
results,  an  adult-leamer/instruction  interaction  motivation  model  has  been  developed 
(see  Figure  1).  This  model,  based  in  part  on  Keller's  (1983)  work,  shows  the 
suggested  interaction  of  methods,  conditi.ons,  and  outcomes  for  the  motivational 
instruction  of  adults.  Methods  are  organized  as  to  the  suggested  effects  on  appeal, 
effort,  and  satisfaction. 

This  model  is  a  beginning  step  in  the  development  of  prescriptions  for  the 
instructional  motivation  of  adult  learners  and  for  the  organization  of  research  in 
adult  motivation.  Because  this  study  only  looked  at  adult  learners  in  graduate  level 
classes  and  continuing  education  workshops,  these  results  may  not  be  generalizable 
to  other  adult  learners.  Further  research  is,  therefore,  needed  to  investigate  the 
perceptions  of  adult  learners  in  other  types  of  instructional  settings.  Research  to 
investigate  the  actual  effects  (as  opposed  to  perceived  effxts)  of  instructional 
strategies  on  the  interest,  effort,  and  performance  of  adult  learners  is  also  indicated. 

The  motivation  of  adult  learners  was  investigated  in  relation  to  interest  and 
effort.  Research  looking  at  further  affective  outcomes  of  these  strategies  is 
suggested.  Other  important  outcomes  which  should  be  investigated  would  include 
learners'  attitudes,  feelings,  values,  and  emotions. 

Research  across  all  age  groups  is  needed  to  investigate  any  differences  in  the 
instructional  motivation  needs  of  learners  of  different  ages.  While  the  literature 
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suggests  that  adult  learners  have  different  needs  than  other  learners,  there  appears  to 
be  little  research  to  support  those  contentions.  To  the  extent  that  this  data  is 
generalizable  to  other  adults,  this  model  can  be  used  by  designers  of  insiruciion  for 
adult  learners.  Specific  strategies  can  be  used  during  the  instructional  process  to 
enhance  the  motivational  elements  of  insiiiiction. 
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Table  1 


Paraphrased  Needs  Assessment  items  Broken  Down  by  Subscaie 


Attention 


Instructor  makes  me  feel  enthusiastic  about  subject 
Class  captures  my  attention 
Instructor  uses  humor 

Instructor  makes  n:e  feel  curious  about  subject  matter 
Instructor  docs  unusual  or  surprising  things 
I  get  a  chance  to  actively  participate 
Interesting  variety  of  teaching  techniques  are  used 
Curiosity  is  stimulated  by  questions  and  problems 
Relevance      Item  1    Learning  will  be  useful  to  me 

Instructor  makes  subject  seem  important 
I  can  see  how  content  is  related  to  what  I  already  know 
I  can  sci  and  achieve  high  standards  of  excellence 
Positive  role  models  are  presented 
Instructor  is  flexible  to  meet  my  needs 
Personal  benefits  are  made  clear 
The  challenge  level  is  not  too  easy  or  too  hard 
Amount  of  work  is  appropriate 
I  have  input  in  content  and  assignments 
I  get  a  chance  to  work  with  other  people 
Content  relates  to  my  expectations  and  goals 
Confidence     Item  1    Instructor  helps  me  feel  confident 

Instructor  helps  feel  success  isn't  linked  to  luck 
Instruction  doesn't  threaten  my  self-esteem 
Whether  or  not  I  succeed  is  up  to  me 
Instructor  creates  relaxed  atmosphere 
Instructor  allows  for  practical  applications 
Requirements  for  success  are  made  clear  to  me 
There  are  frequent  opportunities  to  succeed 
Instructor  links  success  to  my  goals 
Instructor  helps  me  feel  I  can  succeed  with  effort 
Instructor  models  and  demonstrates  proper  skills 
The  instruction  is  non-threatening 
Satisfaction    Item  1    The  class  gives  me  a  lot  of  satisfaction 

The  recognition  I  receive  is  fair  compared  to  others 
I  will  benefit  from  the  knowledge  acquired 
My  instructor's  evaluations  match  my  perceptions 
I  am  not  disappointed  with  the  course 
I  get  enough  recognition  from  the  instructor  for  my  work 
I  feel  satisfied  with  what  I  learn 
I  get  enough  timely  feedback  to  know  my  progress 
I  benefit  from  this  class 
Instruction  is  designed  so  everyone  can  succeed 
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Table  2 

Means  and  Standard  Deviations  of  Responses 
to  Each  Item  in  Course  Interest  Survey  (CISR)  Subscales 


Attention  Relevance 

Confidence 

Satisfaction 

Item  1 

4.81  (.40)        4.84  (.40) 

4.47  (.62) 

4.61  (.58) 

Item  2 

4.12  (1.08)      4.39  (.64) 

4.30  (.92) 

4.24  (.80) 

Item  3 

3.70  (.96)        4.05  (.94) 

4.48  (.79) 

4.75  (.45) 

Item  4 

4.47  (.60)        4.55  (.58) 

3.87  (1.01) 

4.39  (.66) 

Item  5 

4.21  (.72)        4.10  (.74) 

4.43  (.65) 

4.32  (.88) 

Item  6 

4  41  (71) 

4.21  (.72) 

Item  7 

4.32  (.77)        4.26  (.72) 

4.56  (.59) 

4.33  (.70) 

Item  8 

4.51  (.64)        4.17  (.82) 

4.38  (.63) 

4.59  (.58) 

Item  9 

4.40  (.68) 

3.28  (1.02) 

4.52  (.78) 

Item  10 

4.02  (.81) 

4.20  (.76) 

3.77  (1.04) 

Item  11 

3.78  (.86) 

4.37  (.69) 

Item  12 

4.48  (.62) 

4.47  (.73) 

Key: 

1  =  very  negative  effect 

2  =  slightly  negative  effect 

3  =  no  effect 

4  =  slightiy  positive  effect 

n  =  161 

5  =  very  positive  effect 
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Table  3 

Means  and  Standard  Deviations  of  Responses 
to  Each  Item  in  Course  Effort  Survey  (CE3R)  Subscales 


Attention 

Relevance 

Confidence 

Satisfaction 

Item  i 

4.69  (.80) 

4.71  (.68) 

4.41  (.76) 

4.38  (.85) 

Item  2 

4.08  (.96) 

4.10  (.78) 

4.19  (.94) 

4.08  (.83) 

Item  3 

3.63  (.89) 

3.98  (.93) 

4.34  (.87) 

4.55  (.68) 

Item  4 

4.29  (.67) 

4.41  (.76) 

4.06  (.90) 

4.25  (.72) 

Item  5 

4.02  (.74) 

3.84  (.73) 

4.22  (.69) 

4.18  (.91) 

Item  6 

5.01  (.oi) 

4.1,)  V«/4; 

-3  q,,  ( 70"^ 

Item  7 

4.15  (.78) 

4.06  (.76) 

4.30  (.84) 

4.29  (.76) 

Item  8 

4.36  (.69) 

4.12  (.90) 

4.13  (.82) 

4.27  (.69) 

Item  9 

4.18  (.75) 

3.19  (.92) 

4.41  (.83) 

Item  10 

3.86  (.87) 

4.05  (.76) 

3.63  (1.00) 

Item  11 

3.44  (.86) 

4.12  (.73) 

Item  12 

4.17  (.72) 

4.24  (.97) 

Key: 

1  =  very  negative  effect 

2  =  slightly  negative  effect 

3  =  no  effect 

4  =  slightly  positive  effect 

n  =  126 

5  =  very  positive  effect 
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Table  4 


Rankings  of  Effort  and  Interest  Needs  Assessment  Items 


Rankings  Rankings 


Category       Item    Effort  Interest  Category      Item     Effort  Interest 


Attention 


Relevance 


1 

2 

I 

Confidence 

1  * 
1 

iZl 

2 

28t 

33 

L 

1  S 

zo 

3 

39t 

42 

1 

1  n 

4 

12t 

1 

12t 

4* 

"^8 

JO 

5 

33 

29t 

e 

J 

1  / 

1  e 
1 J 

6 

36 

37 

0* 

ot 

1 C 

7 

22 

23t 

7 

11 

6 

8 

8t 

9 

8 

23t 

20 

9 

42 

41 

1 

1 

1 

10 

32 

31 

2* 

27 

18t 

11 

25t 

21 

34 

35 

12 

16 

12t 

4 

4t 

7 

5 

38 

34 

Satisfaction 

1 

7 

4 

6 

23t 

23t 

2 

28t 

28 

7 

30t 

27 

3 

3 

3 

8* 

25t 

32 

4 

15 

18t 

9 

19t 

17 

5 

19t 

23t 

10 

37 

36 

6* 

35 

29t 

11 

41 

39 

7* 

12t 

22 

12* 

21 

lOt 

8* 

14 

5 

9 

4t 

8 

10 

39t 

41 

t  =    tied  rankings 

*  =   large  difference  in  ranking  (above  third  quartile) 
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Naval  Technical  Training  System  Page  1  of  3  Pages 


NAVAL  TECHNICAL  TRAINING  SYSTEM 


1.  The  U.  S.  Navy  is  constantly  engaged  in  technical  training  as  technology  is 
applied  to  ships  at  sea  and  warfare  in  general.     The  entrance  into  the  Navy  by 
young  people  each  year  requires  basic  technical  training  followed  by  advanced 
technical  training  as  these  sailors  move  up  the  ranks  in  their  career, 

2.  Training  requirements  are  derived  from  the  need  for  people  with  particular 
skills  in  the  Fleet.  Skill  requirements  are  maintained  by  the  Chief  of  Naval 
Operations  (CNO)  OP-01  in  the  Enlisted  Billet  File,  which  lists  the  ratings 
and  their  associated  Naval  Enlisted  Classification  (NEC)  Code.  Formal  courses 
dre  taught  in  each  skill  area  unless  enough  people  with  that  skill  are  already 
in  the  trained  inventory  or  it  is  decided  that  skill  may  be  obtained  most  cost 
effectively  fbrough  on-the-job  training  (OJT). 

3.  The  Navy  currently  provides  most  initial  skill  training  via  formal  courses 
and  almost  all  NEC  training  is  provided  by  formal  Class  C  courses.     Old  courses 
are  deleted  when  the  skill  requirements  become  obsolete,  usually  through  the 
process  of  introducing  new  technology.     New  courses  are  introduced  to  meet  the 
new  skill  requirements  resulting  from  the  purchase  of  new  systems  or  equipment. 

4.  The  Navy  Training  Plan  (NTP)  process  is  the  current  mechanism  for  the  intro- 
duction of  new  courses.     This  formal  process  is  as  follows:     First,  someone 
(usually  a  systems  command)  determines  the  need  for  training.    They  define  the 
skills  that  are  required  to  operate  or  maintain  a  new  system  or  piece  of  equip- 
ment;   secona,  this  need  is  followed  by  technical  program  data  on  the  system  or 
piece  of  equipment  in  question;     third,  the  CNO  convenes  a  Navy  Training  Plan 
Conference  attended  by  fleet  representatives,  the  concerned  warfare  desk, 

OP-Ol ,  representatives  of  the  training  community  and  other  concerned  bureaus 
or  commands;     fourth,  the  NTP  is  reviewed,  validated,  approved  and  distributed; 
and  finally,  the  NTP  is  implemented.    NTPs  are  supposed  to  be  developed  and 
approved  at  least  three  years  before  the  introduction  of  new  equipment  into  the 
fleet. 

5.  The  Chief  of  Naval  Education  and  Training  (CNET)  is  responsible  for  provid- 
ing policy,  direction  and  overall  management  for  initial  recruit  and  most 
advanced  specialized  training.     There  are  within  CNET  three  functional  commands 
with  operational  responsibility  for  different  spheres  of  recruit  and  specialized 
training.     The  Chief  of  Naval  Technical  Training  (CNTECHTRA)  in  Memphis  manages 
all  recruit  training,  most  initial  skill  and  skill  progression  training  and 
some  functional  training.     The  Commander,  Training  Command  Atlantic  (COMTRALANT) 
and  Commander,  Training  Command  Pacific  (COMTRAPAC)  manage  a  few  initial  skill 
and  skill  progression  courses  but  are  primarily  concerned  with  courses  of  short 
duration  which  are  usually  operational  in  nature. 
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6»    Technical  training  in  the  Navy  is  a  responsibility  of  the  Chief  of  Naval 
Technical  Training  (CNTECHTRA)  located  at  Naval  Air  Station,  Memphis,  Tennessee. 
CNTECHTRA  Training  Program  Coordinators  (TPC)  are  re&ponsible  for  all  training 
for  specific  ratings  or  groups  of  ratings*    They  compare  the  desired  input 
with  the  training  capacity  of  various  training  facilities  and  identify  resource 
problems  (other  than  manpower)  i.e.,  equipment  or  space  constraints. 

7.    Training  Facilities  (TF)  under  CNTECHTRA  teach  existing  formal  courses  as 
listed  in  the  Catalog  of  Naval  Training  Courses  (CANTRAC).    The  trainees  coroe 
to  the  school  from  a  variety  of  sources  and  include  officers,  enlisted  and 
civilian  personnel  of  the  Navy  and  other  military  services,  including  those 
from  other  nations.    These  facilities  are  shown  on  the  map  below. 
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Naval  Technical  Training  System 


Page  3  of  3  Pages 


LOMG  RANGE  DEVELOPMENT  PROCESS 


1.     Introduction:     The  currjculum  development  process  for  Navy  Technical  Train- 
ing begins  about  three  years  before  the  instruction  of  the  course  is  scheduled 
to  begin.    The  process  is  diagramed  below  to  show  the  normal  flow  of  development 
which  may  be  changed  to  meet  changing  conditions  in  equipment  or  systems 
delivery  to  the  Navy. 


ACTION 
REOUIRED 


Needs 
Assess- 
ment 


PPP  Table 
Development 


TPS 
Development 


Training 

Materials 

development 
^stem 


ACTION 
TAKEN 


Funding 
and 

Billets 


TASK 
ANALYSIS 


JOB 
ANALYSIS 


Stages  1-3 

PLANNING, 

DEVELOPMENT 


Stages  4-5 

VALIDATION, 

IMPLEMENTATION 


PRODUCT 
PRODUCED 


NAVY 

TRAINING 

PLAN-3yrs 

previous 


PPP  Table 
Completed 


TLA 
Completed 


Training  Materials  Completed 

-  Instructor  Guide 

-  Trainee  Guide/Test  Materials 

-  Instructional  Media  Materials 


2.     The  Training  Materials  Development  System  (TMDS)  consists  of  five  stages 
as  shown  below.     Each  stage  is  completed  by  the  Materials  Production  Activity 
(MPA)  which  may  be  a  Training  Facility  (TF)  producing  the  materials  on  an 
in-house  basis  or  utilizing  a  contract  with  an  outside  contracting  production 
organization.     The  second  stage,  developing  sanple  materials,  is  often 
waived  when  the  TF  or  contractor  has  had  previous  satisfactory  materials 
development  experience. 


FIVE  STAGES  OF  T^E  TRAINING  MATERIALS  DEVFLOPhfENT  SYSTEM(TMDS) 
Stage  I :     Plan ,  schedule ,  outline  training  materials 
Stage  2:     Develop  sample  training  materials  unless  waived 
Stage  3:     Complete  draft  development  of  all  training  materials 
Stage  4:    Hold  a  pilot  course  for  validation  of  training  materials 
Stage  5:     Promulgate,  distribute  materials  and  begin  surveillance 
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TRAINING  MATERIALS  DEVELOPMENT  SYSTEM  OUTLINE 

1.  Introduction.     The  Training  Materials  Development  System  (TMDS)  is  a  five 
stage  process  by  which  curriculum  developers  operate,  within  current  regulations 
and  instructions,  to  provide  materials  for  Navy  training. 

2.  Methods  of  training  materials  development. 

a.  Factory  training  development 

b.  Contracting-out  development 

c.  In-house  development 

3.  Documents  utilized. 

a.  Military  Standard:  Contract  Training  Programs  MIL-STD-1379C  (NAVY) 

b.  Military  Handbook:  Training  Materials  Development  DOD-HDBK  292 

c.  Training  Materials  Acquisition  and  Management  Plan  NAVEDTRA  3800AA 

d.  Development,  Review  and  Approval  of  New  or  Modified  Training 
Curricula  OPNAVINST  1550.8B 

e.  Testing  and  the  Measurement  of  Trainee  Achievement  CNTECHTRAINST  1540. 38B 

4«    Management  Participants 

a«  Curriculum  Control  Authority  (CCA) 

b.  Course  Curriculum  Model  Manager  (CCMM) 

c.  Training  Facility  (TF) 

d.  Material  Preparing  Activity  (MPA) 

e.  Training  Program  Coordinator  (TPC) 

f.  Materials  Change  Activity  (MCA) 

g.  Materials  Support  Activity  (MSA) 

5.  Establishment  of  need  for  course 

a.  Navy  Training  Plan  (NTP)  -  Factory  training  curriculum 

b.  Training  Project  Plan  (TPP)  -  Existing  equipment/system 

6.  Process  of  curriculum  development  -  five  stages. 

a.  Pre-development  management  documents  -  NETPMSA  See  page  6 

(1)  Personnel  Performance  Profile  (PPP)  tables  See  page  7 

(2)  Training  Path  System  (TPS)  components  See  page  8 

b.  Stage  1:     Scheduling  and  planning  -  deliverables  Se*^  oage  9 

(1)  Topical  Outline  (TO)  with  Course  Learning  Objective  (CLO) 

(2)  Topic  Learning  Objectives  (TLO) 

(3)  Testing  procedures 

(4)  Preliminary  Master  Materials  List  (MML) 

(5)  Preliminary  PPP-item-to-topic  objective  Assignment  Chart  (OAC) 

(6)  Curriculum  a^aterials  status  and  development  schedule 

(7)  Flow  chart  diagram  of  Stage  1  See  page  10 
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Outline  (continued) 

6.     Process  of  curriculum  development  -  five  stages  (continued) 

c.  Stage  2:     Cross  sectional  sample  of  curriculum  materials 

(1)  Section  of  Instructor  Guide  (IG) 

(2)  Section  of  Trainees  Guide  (TG) 

(3)  Sample  of  Instructional  Media  Materials  (IMM) 
(A)  Sample  Test 

Note:     this  stage  is  usually  waived  for  in-house  development 

d.  Stage  3:     Complete  draft-development  of  curriculum  -  deliverables 

(1)  Instructor  Guide  (IG) 

(2)  Trainee  Guide  (TG)  or  seperate  Instruction  Sheets        See  page  12 

(3)  Instruction  Media  Materials  (IMM)  masters  See  pages  13-16 

(4)  Proctor  Guides  and  Testing  Plan 

(5)  Preliminary  Master  Schedule 

(6)  Flow  chart  diagram  for  Stage  3  See  page  17 


d.  Stage  4:     Pilot  cours     validation  of  curriculum  -  components 

(1)  Purpose  of  Conducting  a  Pilot  Course 

(2)  Preparing  for  Pilot  Course  Convening 
vJ)  Pre-pilot  Conference  Agenda 

(4)  Pilot  course  Monitoring  Team  Responsibilities 

(5)  Conducting  the  Pilot  Course 

(6)  Pilot  course  Final  Report 

(7)  Flow  chart  diagram  for  Stage  4  •  See  page  19 

e.  Stage  5:     Authorized  Curriculum  Managed  and  Modified         See  page  20 

(1)  Promulgation,  production  and  distribution  of  materials 

(2)  Course  implementation  at  Training  Facilities  See  page  20 

(3)  Nsvy  training  appraisal  system  See  page  21 

(4)  Curriculum  materials  modification  t-ystem  See  page  22 

Note-  The  process  of  curriculum  modification  follows  some  of  these 
same  steps,  particularly  when  a  ^'Revision''  of  the  curriculum 
is  required. 
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Training  Materials  Development  System 
PRE-DEVELOPMENT  MANAGEMENT  DOCUMENTS  REQUIRED 

A.  INTRODUCTION.  This  is  a  summary  of  the  documentiS  required  as  the  basis  to 
begin  stage  one  of  the  Training  Materials  Development  System  (TMDS).  The 
references  utilized  to  develop  these  pre-development  documents  are  also  listed. 

B.  REFERENCES. 

1.  NAVEDTRA  38004  REV  A 

2.  Military  Standard     1379C  (NAVY)  with  Data  Item  Descriptors  (DID) 

3.  DOD  Handbook  292,  Parts  1  and  2 

4.  NFTPSMA  Ouarterly  Report 

C.  INFORMATION.     Usually  there  is  a  Navy  Training  Plan.     Two  additional  major 
documents  are  required  in  order  to  effectively  develop  course  materials. 

1.  Personnel  Performance  Profile  Tables  (PPP) 

a.  The  Personal  Performance  Profile  is  a  behavior-oriented  tabular 
listing  of  skills  and  knowledge  to  support  a  subject.     A  PPP  table  consists  of 
a  series  of  PPP  items  derived  from  job  and  task  analysis. 

b.  Five  types  of  PPP  tables  are  used  for  the  follov/ing  purposes: 

(1)  System  (made  up  of  several  sub-systems) 

(2)  Sub-system  (made  up  of  several  related  items  of  equipment) 

(3)  Equipment  (including  related  accessory  items) 

(4)  Task/Function  (dealing  with  one  task  •  r  function) 

(5)  Background  (generalized  academic  information) 

2.  Training  Path  Systetn  (TPS) 

a.  The  Training  Path  System  is  used  by  management  to  determine  which 
PPP  items  were  selected  by  the  Subject  Matter  Expert  (SME)  to  form  the  basis  of 
a  course  and  determine  at  what  "level"  of  training  the  PPP  items  will  be  taught. 
Also,  the  relationship  of  the  new  course  to  all  other  rel atea  courses  for  a 
particular  Navy  Enlisted  Classification  (NEC)  or  oirficer  training  program. 

b.  The  Training  Path  System  consists  of  the  following  components: 

(1)  Training  Objective  Statements  (TOS)  which  modify  PPP  items 
to  indicate  the  level  of  training  for  each  of  three  "task  sets"  representing 
three  types  of  Navy  personnel.     Codes  used  indicate  levels  to  instructors. 

(2)  Training  Level  Assigr:.,2nt  (TLA)  is  a  matrix  showing  the  TOS 
levels  selected  for  each  PPP  item  previously  selected. 

(3)  Training  Path  Chart  (TPC)  comprising  three  "tables": 

(a)  Table  Assignment  Matrix  (TAM^ 

(b)  Table  index  of  PPP  tables 

(c)  Table  Assignment  Chart  (TAC) 

3.  In  the  event  that  these  docut>ents  are  not  provided  by  the  Curriculum 
Control  Authority  (CCA)  and  are  not  available  from  NETPSMA,  guidance  from  the 
Training  Program  Coordinator  at  the  Chief  of  Naval  Technical  Training. 
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4.     PPP  tables  are  developed  from  job  analysis  and  task  analysis  efforts. 


ihe  definition  of  Personnel  Performance  Profiles  (PPP)  tables  is 
as  follows: 

A  list  of  knowledge  and  skill  requirements  for: 

1.  the  operation  and  maintenance  of  a: 
a*  system 

h.  sub-system 

c.  item  of  equipment, 

2.  performance  of  a  t,       or  function,  or 

3.  generalized  academic  background  information 


5.     PPP  tables  are  used  in  curriculum  development  as  the  basis  for: 


1.  the  design,  development  and  management  of  training,  and 

2.  the  development  of  these  curriculum  development  docu;r.snts: 
a.     Training  Path  System  (TPS) 

(to  show  how  the  course  fits  into  total  training) 
0.     Topic  Outline  (TO) 

(to  provide  an  overview  of  the  course  scope  and  sequence) 

c.  Topic  Learning  Objectives  (TLO) 

(to  show  the  specific  knox^ledge  and  skills  to  be  learned) 

d.  Test  Items 

(to  indicate  methods  of  trainee  assessment) 

e.  Instructional  Media  Materials  (IMM) 

(to  provide  the  instructor  and  trainee  with  learning  aids) 


6.     PPP  tables  for  equipment,  subsystems  and  systems  are  organized  according 
to  the  following  outline: 


1  -  KNOWLEOCE  (THEORY;    Describing  the  equipment,  system,  sub-system. 

1-1  General  Desciption 

1-2  Physical  Description 

1-?  Functional  Description 

1-A  Interface  Description 

1-5  OPERATIONAL  DESCRIPTION  (directly  related  to  skills) 

1-6  MAINTENANCE  DESCRIPTION  (directly  related  to  skills) 

1-  7  Documentation  Description 

2  -  SKILLS  (PERFORMANCE)    Listing  the  performance  to  be  accomplished. 

2-  ^1  OPERATION 
2-2  MAINTENANCE 
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7.  The  Training  Path  System  (TPS)  components  can  he  diagramed  as  follows: 

TRAINING  PATH  SYSTEM  (TPS)  COMPONENTS 
TPS 


1  

T  0  S 
Training 
Objective 
Statements 

I 

by  the 
Task  Set — 


T  A  M 
Table 

Assignment 
Matrix 


T  P  C 
Training 
Path 
Chart 


P  P  P 
Table 
Index 


TAG 
Table 

Assignment 
Chart 


TLA 

Training 

Level 

Assignment 

I 

by  the 
-Task  Set 
or  TOS  TO 


8*    The  three  Task  Sets  are  Coordinate,  Direct  and  Perform  and  each  deal  with 
different  Navy  personnel.     These  task  sets  are  diagramed  below,  using  TOS  codes. 


COORDINATE  TASK  SET  (TOS  Codes) 
(Commanding/Executive  Officers) 


DIRECT  TASK  SET  (TOS  Codes) 
(Officers  except  CO/XO) 


0] 

r  (Skill) 

01 

[ 

02 

Ml 

[ 

(Knowledge ) 

F  TO 

Tl 

lo 

F  TO 

T] 

io 

T2o 

T2m 

PERFORM  TASK  SET  (TOS  Codes) 
(Technicans) 


(Skill) 

01 

02  1 

PI 

[ 

CI 

[ 

Li 

(Knowledge)        F  TO 

T] 

io 

T2o 

T2p 

T2c 

t: 

ic 

Training  Objective  Statements  (TOS)  differ  for  each  of  three  Task  Sets. 
Thus  the  Tlo  (Theory  level  j_  for  Operation)  training  objective  statements  for 
the  Coordinate,  Direct  and  Perform  Task  Sets  will  each  be  different  statements. 
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Training  Materials  Development  System 
STAGE  ONE  CURRICULUM  DELIVERABLES 

A.  INTRODUCTION.  This  is  a  summary  of  the  materials  required  for  stage  one 
deliverables  and  provides  the  document  references  for  each  item. 

B.  REFERENCES. 

1.  NAVEDTRA  3800A  REV  A 

2.  Military  Standards     1379C  (NAVY)  with  Data  Item  Descriptors  (DID) 

3.  DOD  Handbook  292,  Part  2 
A.  CNTECHTPAIMST  15A0.38B 

C.  INFORMATION.     The  basic  list  of  stage  one  requirements  is  found  in 
NAVEDTRA  3800A  REV  A,  Chapter  A,  paragraph  A-3.2  a  through  g. 

1.  Course  Learning  Objective  (CLO)  (included  in  Topical  Outline) 

This  multiple-statement  objective  indicates  what  the  trainee  will  be  able  to 
describe  and  perform  when  the  course  has  been  successfully  completed. 

2.  Topical  Outline  (by  part,  section  and  topic) 

Sequencing  of  the  total  course  into  sections  and  topics  to  identify  the  scope 
and  flow  of  the  material.     A  statement  of  rationale  is  included  for  each 
section  and  topic  to  support  inclusion  into  the  course. 

3.  Topic  Learning  Objectives  (TLO) 

A  series  of  statements  forming  the  "backbone"  of  the  course  which  indicate 
what  the  trainee  will  describe  and  perform  upon  completion  of  the  topic. 
Topic  Learning  Objectives  are  composed  for  three  parts: 

(a)  Behavior  -  from  the  PPP  line  item 

(b)  Condition  -  from  the  TOS  training  level  phrase 

(c)  Standard  -  either  stated  or  implied  to  be  100% 

4.  Preliminary  PPP-Item-tu-Topic  Objective  Assignment  Chart  (OAC) 

A  matrix  format  to  relate  each  PPP  line  item  to  the  Part-Section-Topic  of  the 
Instructor  Guide  (IG),  the  Topic  Learning  Objectives  of  the  topic,  and  the 
individual  test  items  which  are  used  to  evauate  each  topic  learning  objective. 

5.  Tests  and  testing  procedures 

A  draft  of  the  procedures  to  be  used  in  trainee  evaluation  during  the  course. 
May  include  some  sample  knowledge  test  items  and  performance  checklists. 

6.  Preliminary  Master  Material  List  (MML) 

A  draft  list  of  all  the  publications,  materials,  equipment,  tools,  expendables, 
etCc  that  are  required  to  teach  the  course. 

7.  Curriculum  Materials  Status  and  Development  Schedule 

A  schedule  to  show  the  start  and  completion  of  each  stage  of  curriculum  develop- 
ment by  year,  month  and  day  as  it  is  planned  by  the  curriculum  developer. 
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FLOW  DIAGRAM  FOR  CURRICULUM  DEVELOPMENT  -  STAGE  ONE 


TRAINING  AGENCY  (TA) 
or 

CURRICULUM  CONTROL 
AUTHORITY  (CCA) 


+Initiate  Tasking 
Letter  to  TF  to 
assign  as  MPA 

+Forward  PPP/TPS 
to  MPA 

+Assign  target  date 
for  delivery  of 
Stage  One  materials 


++Review  and  approve 

development  schedule 
^++Review  Stage  One  materials 
++Coordinate  comments  and 
recommendations  from  TPs 
and  TYCOMs 
++Direct  MPA  to  incorporate 
modifications  in  Stage  2 
or  Stage  Three  materials 


t 


TRAINING 
FACILITIES  (TF) 


Review 
Stage  One 
Materials 


MATERIALS  PREPARING 
ACTIVITY  (MPA) 
(assigned  TF/CCMM) 


+Develop  Stage  One 

materials 
+Provide  copy  of 


Stage  One  materials 
to  TPs,  TA/CCA  and 
TYCOM. 


■r+Modify  Stage 
One  materials 
to  reflect 
approved 
changes 

TO  STAGE  THRE? 


TYPE  COMMANDER 
(TYCOM) 


+Review  Ltage  One 

curriculum  materials 

+Forward  comments  and 

recommendations  to  TA/CCA 


-10- 


ERLC 


107 

117 


CURRICULUM  DEVELOPMENT  SYSTEM  FOR  NAVY  TECHNICAL  TRAINING 


Stage  Three  Curriculum  Deliverables 


Page  I  of  1  pages 


Training  Materials  Development  Ssystem 
STAGE  THREE  CURRICULUM  DELIVERABLES 


A.  INTRODUCTION.  This  is  a  summary  of  starve  three  materials  to  be  delivered 

to  the  Training  Agency  and  provides  the  referencer.  utilized  to  develop  each  item. 

B.  REFERENCES. 

1.  NAVEDTRA  38004  REV  A 

2.  Military  Standard  1379C  (NAVY)  with  Data  Item  Descriptors  (DID) 

3.  DOD  Handbook  292,  Part  2 

4.  CNTECHTRAINRT  1540. 38B 

5.  CNTECHTRAINST  1540. 8C 

C.  INFORMATION.     The  stage  three  requirements  as  shown  below  are  in  their 
completed  final  draft  form  in  preparation  for  course  validation  in  stage  four. 

1.  Instructor  Guide  (IG)  components 

(a)  Front  Matter:  table  of  contents,  how  to  use,  etc. 

(b)  Parts:  sections,  topics,  discussion-demonstration-activity  pages 

(c)  References:  master  materials  list,  objective  assignment  chart,  etc. 

2.  Trainee  Guide  (TG)  components 

(a)  Front  Matter:  hazard  awareness,  list  of  effective  pages,  etc. 

(b)  Instruction  Sheets:     six  types  xor  different  uses 

3.  Instructional  Media  Materials  (IMM) 

(a)  Used  to  supplement  and  aid  in  learning  and  teaching 

(b)  Six  categories  from  audiovisual  to  team  training 

4.  Testing  plan  components 

(a)  What  examinations  will  be  given  at  which  times  during  the  course 

(b)  Written  examinations 

(c)  Performance  examinations  and  laboratory  work 

(d)  Testing  constraints  and  weighting  criteria  for  computing  grades 

3.  Test  item  types  and  tests 

(a)  Item  types:     True-False,  Multiple-Choice,  Completion,  Matchingv 
Essay 

(b)  Test  types:     Knowledge  and  Performance,  Within-course  and  End--of- 
Course 

6.  Preliminary  Master  Schedule 

Represents  "best  guess"  of  curriculum  developer  as  to  number  of  hours 
needed  for  each  topic  of  classroom,  lab  and  testing  activities.     This  will  be 
checked  during  the  Stage  Four  pilot  course  validation  activity. 
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TRAINEE  INSTRUCTION  SHEKT  MATRIX 

!•    Trainee  instruction  sheets  are  provided  to  assist  the  trainee  in  the 
study  of  the  topic  under  consideration.     The  concepts  of  saving  trainee  time, 
supplementing  training  manuals  under  revision,  providing  guidance  during  class 
notetaking,  etc.  are  aimed  at  helping  the  trainee  learn  as  much  as  possible. 


TYPE 

INSTRUCIIONAL  PURPOSE 

ELEMENTS  INCLUDE: 

ADVANTAGES 

Assignment 
"homework" 

1.  Simplify  trainee 
search  for  relevant 
data  needed. 

2.  Direct  trainee 
efforts  to  complete 
an  assignment. 

Title  describes  content 
area;  TLO  from  the  IG; 
Study  assignment  with 
appropriate  documentation; 
Study  questions  to  test 
application  of  information 

1.  SavciS  trainee 
home  study  time. 
2*  Provides 
initial  experience 
in  use  of  refer- 
ence materials. 

Information 
"supplement" 

1.  Cover  inadequacies 
in  technical  manuals. 

2.  Aid  trainees 
comprehension  of 
subject  matter. 

Title  describes  content 
area;   Introduction  explains 
purpose  &  benefit;  Refer- 
ences list  sources  used ; 
Information       material  at 
level  understood  by  trainee 

Provides  trainee 
with  current 
accurate  supple- 
mentary data  CO 
update  training 
manual  material. 

Problem 

"application 
practice" 

1.  Present  trainees 
with  practical 
problems  to  emphasir-e 
the  fundamentals  of 
logical  thinking. 

2.  Give  practice  in 
knowledge  application 
to  problem  solving. 

Title  describes  problem 
content  area ;  Format : 

1.  Clear  statement  of  the 
problem  to  be  solved. 

2.  Conditions  with  any 
parameters  involved. 

3.  Directions  and 
procedures  for  soluation. 

Permits  troubl3- 
shooting  problem 
soluation  without 
requiring  use  of 
equipment  to 
supplement  the 
regular  laboratory 
exercises. 

Job 

"performance 
eva luation" 

Direct  trainees  to 
utilize  documentation 
as  the  basis  for  a 
step-by-step 
performance  of  a 
practical  job  in 
operating  or  main- 
taining equipment. 

Title  describes  content 
area  of  job;  Introduction 
explains  purpose  concisely ; 
Equipment  list;  Reference 
materials;  Job  Steps:gives 
step-by-step  procedures 
with  self-test  questions. 

Provides  trainees 
with  practice  in 
performance  of  a 
job  as  they  will 
do  in  their 
ultimate  job 
assignment. 

Diagram 
"visualized" 

Provide  trainees  with 
course  material  with- 
out the  need  to  copy 
the  chalkboard  or 
overhead  transparency 
material  which  the 
instructor  deems 
important  to  use. 

Title  describes  subject 
matter  involved; 
Format:  varies,  such  as: 

1 .  Foldout  schematics 

2.  Block  diagrams 

3.  Illustrations 

1.  Saves  time  in 
the  classroom. 

2.  Assures  that 
trainees  have 
accurate  infor- 
mation to  work 
with. 

Outline 
"overview" 

Provide  trainees  with 
outline  of  major 
teaching  points  of  a 
topic. 

Title  identical  with  topic; 
Format : 

1 .  Introduction,eg.TLO's 

2.  Outline  in  key  words 

1.  Guides  trainee 
in  note-taking. 

2.  Provides  an 
overview  of  topic 
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INSTRUCTIONAL  MEDIA  MATERIALS  MATRIX 


TYPE 

DESCRIPTION 

EQUIPMENT 
REQUIRED 

ROOM 
LIGHTING 

ADVANTAGES  | 

AUDIO-VISUAL  PRESENTATION  CATEGORY 

1.  Chalk- 
Doard  or 
Visual- 
Aid 
Panel 
(VAP) 

Slate  /  painted 
wood  or  metal  to 
accent  chalk  oi 
special  VAP 
markers. 

Chalkboard 
or  VAP  hung 
on  the  wall. 
May  be 
magnetized. 

Normal 
room 

lighting 

1.  Always  ready  for  use 
during  presei.  ation* 

2.  Easily  erased  when 
finished  with  material. 

3.  Can  be  prepared  prior 
to  class  and  covered. 

2.  Trans- 
parency 

Transparent 
acetate  or 
plastic  film 
with  opaque 
printing  on  it 

Overhead 
projector 
and  screen 

Room  can 

remain 

lighted 

1.  Instructor  faces  the 
class.     Good  eye  contact 

2.  Flexible  sequencing. 

3.  Instructor  may  add 
to  transparency  during 
presentation. 

3.  Films 

8mm  or  16mm 
motion  picture 
film 

8mm  or  16mm 
motion 
picture 
projector 
and  screen 

Room 
darkened 

1.  Shows  intricate 
operations  in  a  clear, 
simple  manner. 

2.  Special  visual 
effects  may  be  used. 

A.  Video 
tapes 

1/2-inch  or 
3/4-inch  reel 
or  cassette 
video  tapes 

Videotape 
player  and 
video  or  TV 
monitor 

Room  can 

remain 

lighted. 

1.  Records  skills  and 
real  life  situations  for 
playback  and  review. 

2.  Can  be  used  for 
immediate  feedback. 

5.  Slides 

35mm  film 
mounted  in 
2x2-inch 
cardboard 
mounts 

Slide 
projector 
and  screen. 
Audio  player 
unit  with 
synchronizer 
if  required. 

Room 
darkened 

1 .  Inexpensive  to 
produce  and  reproduce. 

2.  Flexible  sequencing, 

3.  Instructor  controls 
pace  of  presentation. 

4.  May  be  synced  to  an 
audio  accompaniment . 

6.  Film- 
strips 

35mm  film  in  a 

continous 

strip 

Filmstrip 
projector 
and  screen. 
Audioplayer 
if  filmstrip 
has  sound. 

Room 
darkened 

1.  Fixed  sequence  of 
presentat5  on. 

2.  Economical  produc- 
tion and  reproduction 
4.  May  be  synced  to  an 
audiotape  player. 

7.  Audio- 
tapes 

Coated  plastic 
ribbon  on  reel 
or  in  cassette 

Audiotape 
recorder/ 
player 

Room  can 

remain 

lighted 

1.  Econor.fical  sound 
recor.iing  and  playback. 

2.  Easily  changed. 

i 
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CURRICULUM  DEVELOPMENT^  SYSTEM  FOR  NAVY  TECHNICAL  TRAINING  

Stage  Fnree  Instructional  Media  Materials  Page  2  of  A  pages 


TYPE 

DESCRIPTION 

EQUIPMENT 
REQUIRED 

ROOM 
LIGHTING 

ADVANTAGES 

AUDIO-VISUAL  PRESENTATION  CATEGORY 

8.  Display 
presentat- 
ion easel. 

Easel  with  pad 
of  newsprint  for 
use  with  markers 
May  be  coated  to 
accept  chalk  and 
magnetized* 

A  supply  of 
markers  and 
chalk. 
Magnets  if 
magnetized. 

Normal 
room 

lighting 

1.  Completed  newsprint 
sheets  can  be  displayed 
on  the  room  walls. 

2.  No  time  required  to 
erase  completed  sheets. 

3.  Can  be  completed 
prior  to  class  &  covered. 

TRAINING  GRAPHICS  CATEGORY 

9.  Charts 

Printed  material 
which  gives 
information  in 
tabular  format 

Opaque 
projector 
and  screen 
when  it  is 

Room 

completely 
darkened 
^\.zn  the 
opaque 
projector 
is  used. 
Room  can 
remain 
lighten 
at  other 
times. 

1.  Easy  to  construct. 

2.  Inexpensive. 

3.  Can  be  drawn  on 
chalkboard  or  VAP 
boards  when  projected. 

A.  Easily  duplicated 

for  trainee  handouts. 
5.  Large  classroom 

visuals  are  often 

produced  or  heavy  paper 

or  cloth. 

10.  Graphs 

Symbolic  drawing 
showing 

relationship  or 
comparisons 

desirable  to 
project  the 
graphic  for 
the  entire 

11. Diagram 

Printed  material 
showing  chrono- 
logical changes, 
distribution , 
components  and 
flow. 

class. 
Otherwise, 
no  equipment 
needed . 

TRAINING  DEVICES  CATEGORY 

12.  Models 

Copy  of  the 
real  object: 
enlarged  or 
reduced  size. 

Usually  no 
equipment 
is  required 

Usually 
the  room 
is  kept 
well 
lighted 
to  aid  in 
seeing 
the 

device. 

1.  Clearly  shows 
relationships  among 
objects  and  parts. 

2.  Can  eliminate 
non-essential  elements 
for  learning. 

3.  Appeals  to  muJtiple 
senses  to  aid  learning. 
A.  Interest  the  trainee 
to  develop  greater 
depth  of  understanding. 
5.  Saves  instructional 
time. 

13.Mockups 

Three- 
diamensional 
specialized 
workinj^  model. 

other  than 
the  device 
itself. 

14.  Cut-a- 
way 

Object  which 
has  been 
sectioned  to 
reveal  internal 
structure. 

15.  Actual 
devi  ce 

Use  of  item  of 
equipment  or 
tool. 

Actual  item 
of  equipment 
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CURRICULUM  DEVELOPMENT  SYSTEM  FOR  NAVY  TECHNICAL  TRAINING 


Stage  Three  Instructional  Media  Materials 


Page  3  of  4  pages 


TYPE 

DESCRIPTION 

EQUIPMENT 
REQUIRED 

ROOM 
LIGHTING 

ADVANTAGES 

PROGRA^tMED  MATERIALS  CATEGORY 

16. 

Programmed 
Text  book 
(PI) 

Printed  text  in 
one  of  seve^'al 
programmed 
formats . 

None 

Normal 
room 

lighting 

1.  Individualized  self- 
study  permits  self- 
pacing  to  meet  the 
individual  trainee's 
learning  style  &  pace. 

17. 

Computer- 
Assisted 
Instruct- 
ion (CAI) 

Trainee-operated 
computer- 
controlled 
random-access 
print-based 
instruction. 

Computer 

terminal  with 
"main  frame" 
computer  or 

microcomputer 

2.  Immediate  feedback  on 
responses  provides  the 
trainee  vith  knowledge 
of  errors  due  to 
misinformation. 

3.  Branching  capability 
permits  both  remediation 

18. 

Inter- 
active 

Video 

Disk 
(IVD) 

Trainee- 
operated 
computer- 
controlled 
random-access 
video-based 
instruction 

Video-disc 
player, 
interface 
unit ,  micro- 
computer & 
color  monitor 

and  advancement  within 
the  course  &  saves  time. 
4.  Use  of  computer  disc 
and  video  disc  provides 
rapid  random  acess  to 
course  data. 

SELF- 

-PACED  MATERIALS  CATEGORY 

19. 

Self-study 

workbook 

(SSWB) 

Printed  work- 
book with 
information 
tests,  etc. 
for  self-study 
by  trainee 

Actual  item 
of  equipment 
or  system 
being 
studied. 

Normal 
lighting 

Permits  training  of 
one  trainee  without  the 
physical  presence  of  an 
instructor,  eg.  on-board 
deployed  ships  or  for 
background  remediation. 

20. 

On-the-job 
Training 
Handbook 
(OJT) 

Instructor 
volume  for 
supervisor. 
Trainee  volume 
to  supplement 
manuals,  etc. 

Self -paced  instruction 
held  on-board  to  learn 
tasks  which  cannot  be 
learned  in  the  formal 
classroom  situation. 

21. 

Technical 

Hands-on 

Training 

System 

Packet 

Trainee  train- 
ing packet  for 
self-study . 
Instructor 
guide  packet. 

Self-paced  ini;tructlon 
on  equipment,  sub- 
system or  system  used 
outside  of  formal  class- 
room situation. 
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CURRICULUM  DEVELOPMENT  SYSTEM  FOR  NAVY  xdCHNICAL  TRAINING 


Stage  Three  Instructional  Media  Materials 


Page  4  of  A  pages 


TYPE 

DESCRIPTION 

EQUIPMENT 
REQUIRED 

ROOM 
LIGHTING 

ADVANTAGES 

TEAM  TRAINING  MATERIALS  CATEGORY 

Exercise 
Controller 
Guide 
(ECG) 

Printed  guide 
for  instructor 
to  use  in 
controlling  an 
exercise,  eg. 
Attack  Center 
software. 

As  needed  to 
perform  the 
specific 
exercise 

Appropr- 
iate to 
the 

equipment 
utilized. 

1.  Provides  instructor 
with  multiple  exercises 
for  team  training. 

2.  Permits  revision  of 
or  adding  new  exercises 
without  the  need  to 
revise  the  IG  for  the 

23. 

Instructor 
Utiliza- 
tion 

Handbook 
(lUH) 

Printed  guide 
to  specific 
simulati-r^n 
equipment  for 
instructors 
and  training 
officers  being 
used  in  a 
designated 
course. 

1.  Provides  instructors 
with  "user-friendly" 
guide  to  operation  of 
simulation  equipment 
needed  in  the:.r  course 
of  inst ructiori. 

2.  Provides  students 
with  a  guide  to  their 
activities  in  the 
course. 

2A. 

Simulator 
Software 
Utiliza- 
tion Hand- 
book (SSUH) 

Printed  guide 
for  simulation 
system 

operators  to 
test,  evaluate, 
and  correct 
deficiencies 
in  simulator 

1.  Provides  system 
operators  with  "user- 
friendly"  descriptions 
and  explanations  of 
software  construction 
and  usage. 

FIVE  STAGES  OF  INSTRUCTIONAL  MEDIA  MATERIALS  (IMM)  DEVELOPMENT 
(Reference:  NAVEDTRA  38004A,  Chapter  5,  p.  25) 

1.  Licflng  of  I>fM  to  be  developed  for  the  course  for  review  by  CCA. 

2.  Cross-section  of  IMM  to  be  developed.     (This  stage  may  be  waived) 

3.  Complete  development  of  all  IMM  in  final  draft  form  for  review. 

4.  Validate  IMM  during  "Pilot"  course  along  with  other  curriculum  materials. 

5.  Deliver  master  copies  of  im  to  the  Course  Curriculum  Model  Manager  (CCMM). 
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FLOW  DIAGRAM  FOR  CURRICULUM  DEVELOPMENT  -  STAGE  THREE 


TRAINING  AGENCY  (TA) 
or 

CURRICULUM  CONTROL 
AUTHORITY  (CCA) 


++Assign  a  IF  to  conduct 

the  Pilot  Course 
^++Review  Stage  Three  materials 
++Coordinate  comments  and 

recommendations  from  TFs 
++Assign  TFs  to  monitor  Pilot 
++Direct  MPA  to  incorporate 

approved  modifications  into 

materials  for  Pilot  Course 


1- 


TRAINING 
FACILITIES  (TF) 
(other  than  MPA) 


Review 
Stage  Three 
Materials 

A 


MATERIALS  PREPARING 
ACTIVITY  (HPA) 
(assigned  TF/CCMM) 


+Complete  development 
of  draft  materials 

+Provide  copy  of 
draft  materials  to 
TFs  and  TA/CCA. 


++Modify  validated 
materials  to  reflect 
approved  changes. 
-H-Reproducfc  and 
distribute 
materials. 


TO  STAGE  FOUR 


TYPE  COMMANDER 
(TYCOM) 
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CURRICULUM  DEVELOPMENT  SYSTEM  FOR  NAVY  TECHNICAL  TRAINING 


Stage  Four  ''ilot  Course  Validation 


Page  1  of  2  Pages 


STAGE  FOUR  PILOT  COURSE  VALIDATION  OUTLINE 

NAVEDTRA  3800AA 

1.    Purpose  of  Conducting  a  Pilot  Course^  Appendix  A 

a.  Assure  the  Curriculum  Control  Authority  that  the  course 
"will  work"  through  the  process  of  validation 

b.  Instruct  other  site  instructors  who  will  return  to  their 
training  facilities  to  teach  the  course 

c*     Gain  input  from  other  site  instructors  who  also  serve  as 

subject  matter  experts 
d.     Produce  teachable  "red  line"  edition  of  materials  so  that 

implementation  can  begin  before  final  publication 

5<,     Conducting  the  Pilot  Course> 

a.  Internal  review  of  materials  by  all  those  present  during 
the  pilot  course 

b.  Instructor  responsibilities  include  personalization  of 
the  course  outlined  in  the  instructor  guide 

c.  Trainee  participation  is  observed  by  the  monitors  to  see 
that  the  materials  actual  meet  the  learning  objectives 

d.  Monitor  activities  include: 

(J)  Attendance  at  the  pre-pilot  conference 

(2)  Be  present  for  all  lecture/lab  sessions 

(3)  Comment,  as  appropriate,  on  curriculum 

(4)  Attend  post-pilot  conference 

6.    Pilot  Course  Final  Report* 

a.  Oatline  of  che  final  report. 

(1)  Administration,  ie.  time  allocation 

(2)  Curriculum  Validation,  ie.  adequate  materials 

(3)  Technical  Validation,  ie.  support  of  TLO 

(4)  Redlined  (annotated)  copy  of  all  materials 

(5)  Minority  report  provisions. 

b.  Report  signed  and  sent  to  Curriculum  Control  Authority  for 
approval /disapproval 

c.  The  Curriculum  Control  Authority  either  approves  or  disapproves 
the  report  as  submitted 
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FLOW  DIAGRAM  FOR  CURRICULUM  DEVELOPMENT  -  STAGE  FOUR 


TRAINING  AGENCY  (TA) 
or 

CURRICULUM  CONTROL 
AUTHORITY  (CcA) 


Coordinate 

Pilot 

Course 


TO  STAGE  FIVE 


Receive  the 
Pilot  Course 
Report 


TRAINING 
FACILITIES  (TF) 


Monitor 
Pilot  Course 

Conduct 
Pilot  Course 


Hold 
Post- 
Pilot   

Conference 


Prepare  Pilot 
Course  Report 


Endorse  and  Forward_^ 
Pilot  Course  Report 


MATERIALS  PREPARING 
ACTIVITY  (MPA) 
(assigned  TF/CC^^M) 


TYPE  COMMANDER 
(TYCOM) 
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COURSE  MANAGEMENT  MATERIALS  INFORMATION 

1.  There  are  two  computer  based  management  tools  utilized  by  all  Navy  training 
commands  for  the  purpose  of  information  storage,  retrieval  and  dissemination: 

a.  Catalog  of  Navy  Training  Courses  (CANTRAC) 

(1)  A  NAVEDTRA  publication  distribuced  on  microfiche  (Vol.  II)  and 
in  hardcopy  (Vols.  I  and  III) 

(2)  A  single  alphabetic  character  in  the  first  position  of  the  Course 
Identification  Number  (GIN)  identifies  the  command  having  Curriculum  Control 
Authority,  i.e.,  "A"  =  Chief  of  Naval  Technical  Training. 

(3)  The  Course  Curriculum  Model  Manager  (CCMM)  is  assigned  responsi- 
bility for  maintaining  specified  courses  for  such  items  as  budgeting  for  the 
distribution  of  curriculum  materials  and  conducting  curriculum  reviews. 

b.  Navy  Integrated  Training  Resources  and  Administration  System  (NITRAS) 

(1)  Computer  reports  (files)  are  used  to  manage  and  support  courses. 

(2)  NITRAS  contains  the  following  files: 

(a)  Master  Course  Reference  File  (MGRF)  which  contains  the  course 
schedules,  trainee  statistics  and  class  utilization  data. 

(b)  Student  Master  File  (SMF)  which  contains  information  on  all 
personnel  undergoing  Navy  training  of  13  or  more  days. 

(c)  Training  Summary  File  (TSF)  which  contains  data  on  the  number 
of  trainees  under  Navy  instruction  and  the  "average  on  board"  at  each  Navy 
training  facility. 

(d)  Active  Support  File  (ASF)  which  contains  data  on  manpower 
authority  and  staff  personnel  assigned  to  each  Navy  training  facility. 

2.  A  third  management  tool  is  used  by  those  Navy  training  facilities  which 
utilize  the  NAVEDTRA  llOA  system  of  training. 

a.     Instructional  Management  Plan  (IMP)    which  defines  the  procedures 
for  administration  and  management  of  instruction.     The  following  elements 
make  up  the  IMP:     Cover  page,  table  of  contents,  forward,  introduction,  course 
management  data,  student  flow  management  data,  student  flow  diagram  and  the 
instructor  qualification  plan. 
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Stage  Five  Course  Surveillance 


Page  1  of  1  Pages 


NAVY  TRAINING  APPRAISAL  SYSTEM 

!•    Training  commands  are  tasked  to  provide  properly  trained  personnel  to  the 
fleet.    To  moritor  this  task  the  Chief  of  Naval  Technical  Training  (CNTECHTRA) 
has  established  a  Training  Appraisal  Program  in  order  to  assess  the  quality  of 
training  and  assist  in  iden    fying  and  correcting  any  training  deficiencies. 

2.     Navy  training  appraisal  includes  both  the  process  of  training  and  the 
product  (course  graduates)  of  training.     Some  parts  of  the  program  examine  one 
of  these  two  aspects  while  other  parts  include  examination  of  both  parts  of 
the  program.     The  diagram  below  defines  three  levels  of  appraisal  and  most  of 
the  specific  parts  of  the  program. 

(both  Process  and  Product)  


Level  3  -  CNET/OPNAV/FLEET  CINCS/TYCOMS 
Fleet  Training  Appraisal  Program  (FLETAP)  -  Surveillance 


NAVSEA/ 

SPAWAR 

Technical 

Training 

Audits 


Level  2  -  CNTECHTRA 

X  Internal  Training  Appraisal  Program  (ITAP)/ 

Surveillance 

Triennal  \ 

Command 

Inspections 

\Level  1  -  Training  Facility/ 

Level  II 

Question- 

Training 

\  CISO  QA  Program  / 

naires 

Mission 

Assistance 

Annual  \. 

(TMA)  Team 

Course  \^ 

/  Trainee 

Reviews 

/  Test 

/  Analysis 

Trainee 

Curriculum 

Ir^Jtructor 

Pipeline 

Development 

Evalu ations 

Feedback 

Manage- 

(CD)Team 

from  Newly 

ment 

Trainee 

Assigned 

Reports 

Training 

Course 

Instructors 

Program 

Critiques 

Coordinator 

Staff 

(process      curriculum/instruction/eval. ) 


CNTECHTRA 

Training 

Appraisal 

Survey 

Team 


(product  =  course  graduates) 
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CURRICULUM  DEVELOPMENT  SYSTEM  FOR  NAVY  TECHNICAL  TRAINING  

Stage  Five  Curriculum  Modification  Page  1  of  1  Page 


CURRICULUM  MODIFICATION  MATRIX 


Three  levels  of  curriculum  modification 

of  Modifications 

Interim  Changes 

Changes 

Revisions 

Recommended  by: 
Initiated  by: 

Instructor 

CCMM  or  TF  CO 

CCMM  CO 

\J\JC\ 

Modifications 
may  include... 

Pen  &  Ink  notes 
Technical  error 

cor re ct  ions 
Urgent  safety 

TLO  add/substract 
Change  of  sequence 
or  X ns  L  FX .   nic u nuci  • 
Remove/insert  pgs . 

Change  CLO,TLO  &/ 
or  course  length 
vjUnucriLb  j\j/o  ' 

Limitations  on 
uiuQiricciLiurio  • 

InO  LpLU,    1  lu  or 
added  resources 
No  addition  to 
course  ler'^th 

IN  0  L.  LU  c  n  a  n  ge  s 
No  additional 
resources  or 
course  length 

INO  xxmxLauxons 
Usually  requires 
PILOT  or  other 
validation  action 

Incorporates 

t"ll*aC*a  T^T*PV^^t1Q 
UllCoC     ^  L  C  V  X  O 

actions : 

none 

All       A                  A  m  — 

Axx  xnterxm 
changes  since 
the  last  change 

A1  1  pVioncToc 

since  the  last 
revision 

Alpha/Numeric 
Identification 

Numeric: 

2-1,  2-2,  etc 

Numeric: 

1 ,  2  ,  3  ,  etc 

Alpha: 

A,  B,  C,  etc. 

Approval: 

Authority. . . . 

TF  CO 

specified  form 

CCMM  CO 

specified  form 

CCA 

letter  of 
promulgation 

Notification  & 

Distribution 

Requirements 

TF  reports  to 
CCMM  in  5  days 
CCMM  reports  to 
all  TPs  within 
10  days 

CCMM  distributes 
modified  material 
to  teaching  TFs 

CCMM  distributes 
modified  material 
to  teaching  TFs 

Note:  Technical  Changes  type  of  modification  not  included  in  the  matrix. 


CURRICULUM  CHANGE  PACKAGE: 

1.  Change  identification  -  Numeric,  eg.     Change  1,  Change  2,  etc. 

used  in  sequence  until  next  revision,  eg.  REV  B  CH-A 

2.  Change  notice  -  specified  format  of  letter  giving  instructions 

for  incorporating  changes,  eg.  remove/replace,  delete,  add 
pages.     Includes  justification. 

3.  Change  pages  -  all  modified  pages  of  the  curriculum  that  hsve  been 

provided  to  replace  existing  pages  or  to  be  added. 

4.  Change  materials  -  IMM  (slides,  transparencies,  etc.)  that  have 

been  modified  or  added  to  the  course  materials  (also  new 
MML  pages,  as  needed) 
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Dr.  LRRRV  BUNTING 

SCOTTSDAl  E  COMMUNITY  COLLEGE 


For  years,  education  has  been  grasping  at  new  technologies  for 
the  ideal  educational  tool.     Many  forms  of  instructional  technology 
have  come  and  gone  in  the  last  fifteen  years.     Some  were  acquired 
on  the  recorr-.endation  of  equipment  vendors .     Others  were  purchased 
with  the  hope  of  being  on  the  lear^  ag  edge  and,   if  successful, 
would  mean  a  feather  in  the  cap  ot  the  individual  who  thought  of 
it.     There  are  problems  with  acquiring  instructional  technology  in 
this  manner.     The  first  example  is  almost  always  sold  by  a  vendor 
who  is  usually  biased  with  the  opinion  of  his  own  product.     In  the 
second  example,  the  individual  has  good  intentions  but  there  is 
invariably  a  good  chance  for  failure. 

Education  via  an  educational  satellite  system  is  not  exempt 
from  these  problems.     This  is  why  the  first  step  in  considering  the 
adoption  of  this  technology  as  an  educational  delivery  system 
should  be  to  better  understand  the  individual  groups  (students, 
faculty,  and  administration)  that  will  be  using  and  served  by  such 
a  system.     The  administration  must  be  considered  because  of  their 
problems  with  obtaining  financial  backing,  curriculum  adoption,  and 
class  scheduling.     The  faculty  must  be  considered  because  they  are 
the  individuals  that  have  to  work  with  the  technology  and  see  that 
the  system  operates  effectively.     The  students  are  the  most 
affected  since  they  are  the  recipients  of  \:he  information  and  are 
the  reason  for  the  existence  of  the  educational  process. 

In  order  to  consider  these  three  groups  and  their  problems, 
research  was  conducted  using  a  group  of  nineteen  community 
colleges.  The  study  looked  at  these  three  groups  in  order  to  get  a 
better  understanding  of  their  attitudes,  perceptions,  and  styles  of 
teaching  in  regard  to  technology,   facilities,  and  level  of 
knowledge  in  var'.ous  areas  that  were  applied  to  distance  learning 
via  satellite. 

In  order  to  get  a  complete  picture  of  the  three  groups,  a 
determination  was  made  as  to  what  information  should  be  gleaned 
from  the  participants  that  would  give  the  appropriate  information 
required  for  making  decisions  for  a  distance  learning  system. 

Along  with  demographic  information,  the  following  were 
investigated  and  the  results  are  reported. 


122 


ERIC 


2 


1^  ^     I:^£2rni.n^  StyXssi     Stucl6nts  snd  faculty  V76rs  not  in  totsl 
agreement  as  to  what  style  was  most  commonly  used  at  their 
institutions.     Both  indicated  that  the  lecture  method  was  the  most 
dominant  but  faculty  felt  that  the  use  of  media  materials  was  in 
greater  usage  than  did  the  students.     The  majority  of  faculty  had 
never  tried  teaching  styles  other  than  lecture  with  the  use  of 
media  materials. 

2.  Lavels  of  Knowledge:  Administrators  and  faculty  were 
questioned  as  to  their  knowledge  in  the  areas  of  institutional 
facilities^  alternative  delivery  systems^  and  satellite 
communications.     In  all  instances,  the  administrators  were  the  most 
knowledgeable.     However,  when  it  came  to  identifying  various  kinds 
of  satellite  terminology,  neither  the  faculty  nor  administrators 
were  very  knowledgeable. 

3.  How  the  Participants  Felt  About  Educational  Technology: 

According  to  the  three  groups,  educational  technology  is  of  great 
value  in  education  and  should  continue  to  be  steadily  emphasized. 
Of  the  three  groups  surveyed,  the  administrators  showed  the 
greatest  support  for  educational  technology. 

4 .  How  Individuals  Felt  About  the  Use  of  Educational 
Satellites:     Using  satellites  for  distance  learning  was  looked 
upon  favorably  by  all  three  groups,  but  none  were  over-zealous 
about  the  idea.     Of  the  three  groups,  the  administrators  were  the 
most  support  ive . 

When  considering  the  use  of  satellites  for  international 
(  ducation,  the  administrators  had  mixed  feelings  regarding  the 
luture  of  satellites  in  international  education. 

5.  The  Level  of  Administrative  Support:    Administrators  were 
very  willing  to  support  educational  technology  in  general  but  were 
found  to  be  spending  very  little  for  alternative  forms  of  delivery. 
Even  though  some  administrators  were  not  budgeting  any  funds  for 
alternative  delivery  systems,  they  still  felt  very  strongly  about 
its  value  for  education. 

6.  How  Faculty  Felt  Toward  Educational  Technology  in 
Relation  to  Their  Teaching  Styles:     Lecturing  and  using  media 
materials  was  the  dominant  style  of  teaching.     Those  faculty  whose 
teaching  style  was  using  lecture  only  were  consistently  less 
convinced  of  the  benefits  of  technology  in  education. 

Faculty  who  taught  by  using  lecture  and  discussion  or  media 
believed  that  faculty  should  be  doing  more  experimenting  with 
educational  technology. 

All  groups,  even  those  that  taught  mainly  by  computer,  felt 
that  at  times  technology  is  moving  too  rapidly  in  our  educational 
system. 

7 .  How  Faculty  Felt  Toward  Educational  Technology  in 
Relation  to  Their  Curriculum:     Two  groups  of  faculty  that  stood 
out  were  those  in  the  Math-Science  field  and  those  teaching  in  the 
Social  Science  curriculum. 
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The  Math-Science  faculty  v;ere  the  most  positive  in  regard  to 
the  use  of  technology  in  education,  while  the  Social  Science 
faculty  did  not  see  it  as  beneficial  for  education  as  did  faculty 
in  the  other  curriculums . 

8.  How  Students  Felt  Toward  Educational  Technology  in 
Relation  to  Their  Major  Curriculums :  Two  groups  of  students 

that  stood  out  were  those  in  the  Math-Science  field  and  those  with 
a  major  in  the  Social  Science  curriculum. 

The  Math-Science  students  were  the  most  positive  in  regard  to 
the  use  of  technology  in  education  whereas  the  Social  Science 
students  did  not  see  it  as  beneficial  for  education  as  did  students 
in  the  other  curriculums . 

9 .  How  Faculty  Felt  Toward  Educational  Satellite 
Communications  in  Relation  to  Their  Discipline:     All  faculty 
represented  in  each  of  the  various  disciplines  were  very  supportive 
of  satellite  education.     Faculty  teaching  Technology  courses  showed 
the  most  favorable  attitude  toward  education  by  satellite.  Faculty 
who  teach  in  the  Health  Science  field  were  the  least  receptive  to 
so:ne  of  the  ideas  about  what  satellites  are  doing  for  education  but 
over  80%  felt  that  teaching  with  satellites  is  an  exciting  concept. 

10.  How  Students  Felt  About  Educational  Satellite 
Communications  in  Relation  to  Their  Major  Curriculiun: 

Students  were  middle-of-the-road  when  it  came  to  indicating  they 
would  take  a  course  via  satellite.     Technology  students  showed  a 
very  positive  opinion  relative  to  the  use  of  satellites  whereas 
Social  Science  students  had  the  least  favorable  opinions. 
Math-Science  students  showed  the  least  resistance  in  taking  a 
course  via  satellite  whereas  the  Communications  students  showed  the 
most  oppos  it  ion . 


Recommendations  Based  on  the  Reaulta  of  thft  Study 

Once  the  information  was  gathered  for  the  above  statements ,  a 
total  picture  of  the  students,   faculty,  and  administrators  became 
clear  and  the  following  recommendations  were  then  proposed. 

1.     Purchase  Satellite  Dish  -    By  doing  this,   it  will  be 
possible  to  demonstrate  to  the  individuals  involved  as  to  what 
information  is  presently  being  sent  from  satellites  at  no  expense 
to  the  downlink  sites  .     This  will  help  with  individuals  who 
expressed  negative  feelings  toward  the  technology  and  at  the  same 
time  take  care  of  some  of  the  hardware  requirements  for  interactive 
instruction  by  satellite . 

2  .     Public  Relations  -    If  the  schools  involved  should  decide  to 
use  satellites  as  a  distance  learning  system,  one  of  the  first 
steps  to  be  taken  is  that  of  informing  the  users.  This 
informational  program  should  include  the  viewing  of  existing 
satellite  programming  and  an  explanation  of  how  interaction  occurs. 
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The  administrators,  faculty,  and  students  should  be  able  to  talk 
with  individuals,  using  a  satellite  system,  in  order  to  obtain 
information  which  is  not  always  covered  in  large  group  meetings  or 
through  informational  literature. 

3,  Budget  -    Many  schools  did  not  report  a  budget  for  alternative 
delivery  systems.     A  sharing  of  information  should  be  undertaken 
with  schools  that  are  presently  using  satellites  for  curriculum 
support  in  order  to  discover  all  areas  requiring  funding  and  the 
possible  hidden  cost  that  might  exist* 

4,  Curricultam  Selection  -    As  stated  earlier,  some  curriculums 
will  lend  themselves  more  readily  to  satellite  technology  than  will 
others*     Each  school  should  conduct  an  investigation  to  find  out 
what  these  curriculums  might  be  and  then  begin  the  use  of  satellite 
technology  with  the  ones  that  show  the  least  resistance.     To  not  do 
this  could  prove  to  be  counter-productive, 

5.  Technology  Integration  ~    If  institucions  decide  to  use  an 
interactive  satellite  system,  they  should  begin  instructing  faculty 
in  the  best  ways  of  integrating  the  technology  within  their  daily 
teaching.     Doing  this  would  help  to  decrease  the  possible 
apprehension  experienced  by  those  faculty  that  would  be  using 
satellites  as  part  of  their  teaching*     Too  many  times  faculty  are 
given  the  hardware  and  material  for  utilizing  technology  but  are 
not  instruct  d  in  how  it  is  to  be  used* 

6.  Services  Available  -    It  was  determined  that  many  of  the 
faculty  were  not  aware  of  facilities  and  services  related  to 
educational  technology  that  presently  existed  at  their  school*  An 
on-going  program  to  inform  faculty  about  what  is  available  should 
begin  as  soon  as  possible  even  though  the  actual  use  of  satellites 
may  not  occur  in  the  near  future. 


As  stated  earlier,  this  study  was  looking  at  a  specific  group 
of  community  colleges  and  the  information  applies  specifically  to 
them.     Even  though  the  findings  apply  to  this  group  of  colleges, 
the  findings  can  be  generalized  for  any  educational  setting* 

The  recommendations  were  drawn  from  the  information  gathered  in 
this  particular  study*     Other  possible  recommendations  could  have 
been  made,  however  those  listed  are  the  ones  believed  to  be  the 
most  crucial  for  identifying  procedures  for  adopting  and  using  an 
educational  satellite  system  as  a  distance  learning  tool*  Further 
study  might  be  of  value  for  identifying  additional  procedures  to 
enhance  the  adoption  and  use  of  satellite  technology* 

One  major  concept  that  has  been  drawn  from  the  study  is  to  not 
let  hardware  determine  the  way  courses  will  be  taught.  Involvement 
of  the  users  from  the  initial  investigative  stages  is  essential 
when  considering  an  educational  satellite  system  as  a  form  of 
instruction. 
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Can  Interactive  Video  Overcome  the  "Couch  Potato"  Syndrome? 

Current  liteature  abounds  with  the  promises  of  Inteactive  video  for 
revolutionaizing  the  (nstructional  process.  Although  DeBloois  (1982)  contended 
that  interactive  video  is  an  entirely  new  medium  with  characteristics  quite 
uniil<e  those  of  computers  and  video,  recommendations  for  design  and 
implementation  of  interactive  video  are  often  based  on  past  televsion  and 
computer  research  (see  for  example:  Hannafin,  Garhart,  Rieber,  & 
Phillips, 1985). 

Salomon  (1983;  1984;  Salomon  &  Leigh,  1984)  conducted  several 
influential  studies  on  learning  from  television,  and  references  to  his  findings  are 
prevalent  in  the  literature  on  interactive  video.  In  the  most  frequently  cited 
study,  Salomon  (1984)  measured  learners'  preconceptions  of  the  ease  or 
difficulty  of  print  and  television,  then  the  learners  received  either  a  print-based 
or  video-based  lesson.  Upon  completion  of  the  lesson,  learners  completed  a 
questionnaire  on  their  perceptions  of  the  amount  of  mental  effort  that  they 
invested  in  ine  learning  task,  and  were  then  given  an  achieve^nent  test.  Salomon 
found  XhQi  students  perceived  television  as  easier  than  print.   Learners  in  the 
print  condition  reported  investing  more  mental  effort  and  had  higher 
achievement  scores  than  students  who  received  the  video  lesson.  Salomon 
concluded  that  the  learners*  preconception  of  television  as  a  easy  medium 
resulted  in  a  decreased  investment  of  mental  effort  and,  consequently,  lower 
achievement  scores. 

Several  authors  (Dalton,1986;  Hannafin,  et  al,  1985;  Schaffer  and 
Hannafin. 1986)  have  suggested  that  the  active  response  required  by  interactive 
video  lessons  may  overcome  learners*  preconceptions  of  television  as  a  easy, 
passive  medium,  and  result  in  increased  mental  effort  and  achievement.  But 
although  the  results  of  Salomon's  studies  with  linear  video  have  been  freely 
generalized  to  interactive  video  research  and  design,  there  is  little  research  that 
has  explicitly  examined  learners'  preconceptions  of  interactive  video. 

A  review  of  cognitive  and  motivational  theory  indicates  that  learners* 
scripts  (Schank  &  Ableson,  1977),  schemata,  attributions  (Weiner,  1979)  and 
curiosity  (Berlyne  1960)  may  influence  their  preconceptions  of  a  medium. 
Further  examination  of  the  literature  suggests  that  learners*  preconceptions  may 
be  influenced  by  their  past  experiences  (Schank  &  Ableson,  1977),  their 
perceptions  of  the  learning  task  (Krendl  &  Watkins,  1983;  Salomon  &  Leigh, 
1984;  Weiner,  1979),  and  the  characteristics  of  the  medium  (Berlyne,  1960). 

The  characteristics  of  the  medium  can  be  manipulated  easily  in  order  to 
alter  the  learner*s  preconceptions  of  a  mediated  lesson  and  increase  the  mental 
effort  that  learners  invest  in  processing  the  lesson.  Several  researchers  (see 
for  example:  Dalton  &  Hannifin,  1987;  Heestand,  1980;  Lielke  &  Gall,  1985) 
have  included  practice  questions  in  video-based  materials  and  found  that  actively 
responding  to  embedded  questions  resulted  in  greater  achievement  scores  than 
viewing  a  video-tape  without  practice  questions.  Anderson  and  Biddle  (1975) 
reviewed  the  literature  on  practice  questions  in  film,  lecture,  and  text  and 
concluded  that  practice  questions  seemed  to  faciliate  the  accquisition  of  both 
practiced  and  incidental  information. 

Although  no  researchers  have  included  practice  questions  in  video-based 
materials  and  examined  the  effects  on  students*  perceptions  of  mental  effort, 
several  researchers  (Britton,  Piha,  Davis,  &  Wehausen,  1978;  Burton.  Niles  & 
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Lalik,  1986;  Reynolds  &  Anderson.  1982)  have  included  practice  questions  in 
text  materials  and  found  that  practice  questions  may  have  increased  the  amount  of 
mental  effort  used  in  processing  the  lessen. 

Purpose  of  the  Study 

The  purpose  of  this  presentation  is  to  summarize  the  results  of  a  study 
that  (a)  investigated  learners'  preconceptions  of  i'  'c^active  video  (IV), 
instructional  television  (ITV),  and  television  (TV),  ano  (b)  compared  three 
treatment  groups  on  learners'  perceptions  of  invested  mental  effort  and 
achievement  on  a  test  of  recall  and  inference.  The  three  treatments  consisted  of 
an  IV  lesson  that  included  practice  questions  requiring  an  active  response,  an  ITV 
lesson  that  included  practice  questions  requiring  a  covert  response,  and  a  TV 
lesson  that  did  not  include  practice  questions.  The  relationship  between  learners' 
preconceptions  and  perceived  mental  effort  was  also  investigated. 

Research  Hypotheses 

The  primary  assumptions  of  tne  current  study  were  based  upon  Salomon's 
(1983;  1984;  Salomon  &  Leigh,  1984)  findings  that  (a)  television  is  perceived 
easy,  (b)  there  is  a  negative  correlation  between  preconceptions  of  the  ease  of 
a  medium  and  the  amount  of  perceived  mental  effort,  and  (c)  perceived  mental 
effort  is  positively  correlated  with  achievement  on  a  test  of  inference  items. 

These  primary  assumptions  were  further  developed  based  on  a  review  of 
the  related  literature  and  the  followinc,  expectations  were  generated:  (a) 
Learners  would  perceive  IV  to  be  more  difficult  than  ITV  and  TV,  and  learners 
would  perceive  ITV  to  be  more  difficult  than  TV;  (b)  learners  would  report 
expending  greater  mentai  effort  in  processing  an  IV  lesson  th^n  in  processing  an 
ITV  lesson  or  a  TV  lesson,  and  learners  would  report  expending  greater  mental 
effort  in  processing  an  ITV  lesson  than  in  processing  a  TV  lesson;  (c)  there  would 
be  a  significant  positive  correlation  between  the  perceived  difficulty  level  of  the 
video  treatments  and  learners'  self-reports  of  the  amount  of  mental  effort 
invested  in  processing  the  lesson  using  that  medium;  (d)  learners  who  received 
the  IV  lesson  would  have  higher  scores  on  a  posttest  of  inferences  than  learners 
who  received  the  ITV  lesson  and  the  TV  lesson,  and  learners  who  received  the  ITV 
lesson  would  have  higher  scores  on  a  posttest  of  inferences  than  learners  who 
received  the  TV  lesson;  (e)  there  would  be  no  significant  difference  in  the  recall 
scores  of  learners  who  received  an  IV  lesson,  an  ITV  lesson,  or  a  TV  lesson;  (f) 
there  would  be  a  sigrHiicant  positive  correlation  between  learners'  estimates  of 
mental  effort  and  achievement  scores  on  a  test  of  inference  items. 

Method 

A  posttest-only  control  group  design  was  used  for  this  study.  The 
independent  variable  consisted  of  the  video-based  instructional  materials  with 
three  levels  (IV  vs  ITV  vs  TV).  The  students  who  received  the  IV  treatment  were 
provided  with  active  practice,  the  students  who  received  the  ITV  treatment  were 
provided  with  covert  practice,  and  the  students  who  received  the  TV  treatment 
were  provided  with  no  practice. 

The  dependent  variables  consisted  of  student  ratings  of  perceived  mental 
effort,  recall  scores,  and  inference  scores.  Information  on  student 
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preconceptions  of  the  difficulty  of  learning  fronr^  IV.  ITV.  and  TV  was  also 
collected  for  descriptive  and  correlational  purposes. 

It  was  assumed  that  random  assignment  controlled  for  differences  in 
prior  knowledge;  however,  a  pretest  was  administered  to  a  cohort  group.  The 
pretest  confirmed  that  the  target  population  did  not  have  extensive  prior 
knowlcylge  of  the  content  of  the  lesson:  The  mean  score  on  a  combined  test  of 
recall  and  inference  items  was  A.Z  points  out  of  a  possible  60  points. 

Participants 

Seventy-eight  undergraduate  students  were  randomly  selected  and 
assigned  to  the  three  treatments;  however,  the  final  sample  consisted  of  23 
students  in  the  IV  group,  25  students  in  the  ITV  group,  and  23  students  in  the  TV 
group  due  to  attrition. 

Instructional  Materials 

The  instructional  materials  consisted  of  a  TV  lesson,  an  ITV  lesson,  and  an 
IV  lesson  of  approximately  20  minutes  in  length.  Segments  from  an  existing 
interactive  video  science  unit  on  the  topic  of  fission  and  fusion  were  modified  for 
use  in  this  study.  The  instructional  strategy  followed  the  events  of  Instruction 
outlined  by  Gagn6  and  Briggs  (1979)  and  was  similar  for  each  treatment. 

The  manner  in  which  the  events  of  practice  and  feedback  were  presented 
in  the  three  lessons  constituted  the  primary  difference  among  the  three 
treatments.  Conventional  television  programs  typically  do  not  provide  practice 
and  feedback  on  the  concept  presented  in  the  prograrr  (Smith  &  Andrews,  1985); 
therefore,  the  TV  treatment  did  not  contain  any  form  of  practice  and  feedback. 
Although  both  the  IV  lesson  and  the  ITV  treatment  included  practice  and  feedback, 
learners  were  required  to  overtly  respond  to  practice  questions  in  the  IV  lesson, 
but  were  only  given  the  opportunity  to  covertly  respond  to  the  practice  questions 
in  the  ITV  lesson. 

Following  the  presentation  of  information  for  each  objective  (verbal 
information:  recall  and  inference),  the  learners  in  the  IV  and  ITV  treatments 
were  presented  with  a  text  screen  that  asked  them  to  identify  all  statements  that 
vvf»re  true  about  the  topic  previously  presented.  In  the  IV  treatment,  students 
were  required  to  touch  all  of  the  correct  phrases  and  their  responses  were 
followed  by  •'knowledge  of  response**  feedback.  In  the  ITV  treatment,  practice 
questions  were  followed  by  an  extended  pause  and  the  students  were  expected  to 
respond  covertly.  In  both  the  iV  and  ITV  treatments,  each  text  screen  that 
contained  a  practice  question  was  followed  by  a  text  screen  that  provided  the 
correct  answers.  The  feedback  in  the  IV  lesson  remained  on  the  screen  until  the 
learners  touched  the  screen  to  continue  the  presentation;  the  feedback  in  the  ITV 
lesson  was  followed  by  an  extended  pause. 

Collected  Data 

Preconceptions  of  difficulty.  A  preconceptions  questionnaire  was 
administered  immediately  prior  to  the  lesson  to  assess  students*  preconceptions 
of  the  difficulty  of  IV,  ITV,  and  TV.  The  questions  were  identical  to  those  used  by 
Salomon  (1984)  to  measure  student  preconceptions  of  print  and  television 
except  for  the  subsitution  of  the  words  **interactive  video\  Instructional 
television**,  and  **television*'  in  place  of  **television**  and  **a  book**.   To  ensure  that 
each  of  the  participants  interpreted  the  terms  **interactive  video^  "instructional 
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television^  and  "television-  in  the  same  manner,  the  terms  were  operationally 
defined. 

Students  were  asked  to  rate  their  responses  to  questions  such  as  -  How 
easy  would  it  be  for  you  to  learn  to  solve  a  math  problem  from  interactive 
video?"  on  a  5-point  Ukert  scale.  One  represented  Very  easy"  and  five 
represented  "very  difficuir.  There  were  three  questions  for  each  medium,  and 
in  accordance  with  Salomon's  methodology,  the  scores  on  the  three  items  for  each 
medium  were  added  together  to  result  in  an  overall  score  for  TV,  an  overall  score 
for  ITV,  and  an  overall  score  for  IV. 

Although  Salomon  reported  a  Cronbach's  alpha  of  .89  for  his 
questionnaire,  in  the  current  study,  the  Cronbach's  alpha  was  .75.  However,  the 
questionnaire  was  assumed  to  have  an  acceptable  degree  of  intemal  consistency 
for  the  purposes  of  this  study. 

Mental  effort.  After  completing  the  instructional  materials,  students 
were  given  a  brief  questionnaire  which  asked  them  to  estimate  the  amount  of 
mental  effort  that  they  had  invested  in  processing  the  lesson.  The  questionnaire 
consisted  of  six  questions  that  were  derived  by  combining  the  questions  that 
Salomon  used  in  two  studies  (1983;  1984).  Students  were  asked  to  rate  their 
responses  to  questions  such  as  "How  hard  did  you  concentrate  while  watching  the 
lesson?".  Students  responded  on  a  five  point  scale  of  effort  with  1  representing 
"low"  and  5  representing  "high".  Responses  to  each  item  were  added  together  to 
provide  a  total  score  for  each  student.  An  analysis  of  the  internal  consistency  of 
the  instrument  used  in  this  study  resulted  in  a  Cronbach's  alpha  of  .55. 

Achievement  measures.  Cued  recall  and  cued  inference  questions  were 
included  in  the  achievement  measure.  The  questions  required  constructed 
responses  that  ranged  in  length  from  one  sentence  to  several  paragraphs.  Student 
responses  were  scored  as  to  the  number  of  correct  facts  or  ideas  that  ttiey 
generated  in  response  to  the  questions. 

The  recall  items  consisted  of  eight  cued  recall  questions  pertaining  to 
explici'iy  presented  verbal  information.  The  test  of  inferences  consisted  of  seven 
open-ended  questions.  Responses  to  the  inference  questions  could  include 
information  that  was  expiicitly  stated  in  the  lesson  as  well  as  inferences  that 
connected  ideas  that  were  explicitly  presented  in  the  lesson.  Responses  that 
restated  information  that  was  explicitly  presented  in  the  lesson  were  coded  as 
"recall";  responses  that  connected  ideas  that  were  stated  in  the  lesson  were  coded 
as  "inferences".  Each  student  received  a  recall  score  that  reflected  the  total 
number  of  facts  that  were  correctly  recalled  in  response  to  the  recall  and 
inference  questions  (maximum  possible  recall  score  «  45)  and  an  inference 
score  that  reflected  the  total  number  of  inferences  generated  in  response  to  the 
inference  questions  (maximum  possible  Inference  score  «  18). 

Using  the  scoring  key,  an  acceptable  interrater  reliability  of  .95  was 
achieved  among  the  researcher  and  three  graduate  students  in  instructional 
technology. 

Additional  data  collected.  In  addition  to  collecting  data  that  was  essential  to 
the  measurement  of  the  dependent  variables,  students  were  asked  to  record  the 
time  when  they  began  the  lesson  and  when  they  completed  the  lesson.  The  time  at 
which  the  student  began  the  lesson  was  subtracted  from  the  time  at  which  the 
student  ended  the  lesson  to  result  in  a  lesson  time  score. 
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Upon  reporting  to  the  research  site,  students  were  expected  to  complete 
the  treatment  independently  and  with  a  minimum  of  supervision-  The 
experimental  packet  contained  general  instructions  on  how  to  proceed  through  the 
lessoP:  Instructions  on  the  operation  of  the  equipment,  and  all  treatment 
materials.  First,  the  participants  completed  the  preconceptions  questionnaire, 
then  they  received  instructions  to  proceed  to  the  IV  table,  the  ITV  table  or  the  TV 
table,  depending  on  the  student's  assigned  treatment.  After  students  completed  the 
lesson,  they  were  instructed  to  complete  the  questionnaire  on  mental  effort. 
After  completing  this  questionnaire,  the  students  were  askbd  to  complete  the 
posttest  of  recall  and  inference  items. 

Results 

The  data  were  analyzed  using  several  statistical  tests  of  significance. 
Analysis  of  variance  (ANOVA)  was  used  to  analyze  the  data  among  groupus  on 
perceived  mental  effort  ratings,  recall  scores,  and  inference  scores.  A  repeated 
measures  ANOVA  was  used  to  determine  if  there  were  significant  differences 
among  learners'  preconceptions  ratings  for  IV,  ITV,  and  TV.  This  test  was  used  to 
analyze  the  preconceptions  data  because  ratings  for  each  of  the  three  media  were 
obtained  from  each  participant,  thus  the  ratings  were  highly  correlated. 
Following  a  finding  of  a  significant  £,  Tukey's  Honestly  Significant  Difference 
(HSD)  test  was  used  to  determine  the  significant  differences  among  individual 
group  means.  Con-elations  among  scores  on  the  measurement  instruments  were 
tested  using  a  f'earson's  i  test  of  correlation. 

Preconceptions  of  Difficulty 

Out  of  a  maximum  possible  difficulty  rating  of  15  for  each  medium,  the 
mean  rating  for  IV  was  5.47,  the  mean  rating  for  ITV  was  6.21,  and  the  mean 
rating  for  TV  was  7.90. 

A  repeated  measures  ANOVA  indicated  that  there  was  a  significant 
difference  in  the  learners'  preconceptions  of  the  difficulty  of  IV,  ITV,  and  TV, 
£(2,140)  =  48.43,  {i  <  .01.  A  Tuke/s  HSD  post-hoc  analysis  (u  <  .05)  of  the 
data  indicated  that  learners  felt  that  it  was  significantly  easier  to  learn  from  IV 
than  from  ITV  and  TV,  and  significantly  easier  to  learn  from  ITV  than  from  TV. 

Perceived  Mental  Effort 

Out  of  a  maximum  possible  perceived  mental  effort  rating  of  30,  the  mean 
rating  for  IV  was  20.56,  the  mean  rating  for  ITV  was  21.04,  and  the  mean  rating 
for  TV  was  20.00.   ANOVA  results  indicated  there  were  no  significant  differences 
in  the  learners'  perceived  mental  effort  ratings  between  treatments,  E(2,68)  = 
.69,  {i  =  .51. 

Preconceptions  of  Difficulty  and  Pernfiivpri  Mental  Effort 

Pearson's  i  test  for  correlation  indicated  that  the  correlation  between  the 
mean  preconceptions  ratings  and  mean  perceived  mental  effort  ratings  did  not 
reach  significance  for  IV,  l=  -.31,  ITV,  l=  -.13,  or  TV,  l=  -.07. 


132 

140 


7 

The  maximum  possible  inference  score  was  18  and  the  mean  score  for  IV 
was  5.04,  the  mean  score  for  ITV  was  4.60,  ar.d  the  mean  score  for  TV  was  4.47- 
ANOVA  results  indicated  that  there  were  no  significant  differences  between  the 
mean  inference  scores  of  the  three  groups,  £(2,68)«  .27,  p.  «  .76;  however  the 
mean  Inference  scores  were  in  the  predicted  order  (IV  >  ITV  >  TV). 

Recall  Scores 

The  maximum  possible  recall  score  was  45  and  the  mean  score  for  IV  was 
20.4,  the  mean  score  for  ITV  was  18.6,  and  the  mean  score  for  TV  was  15.5. 
ANOVA  results  indicated  that  there  were  significant  differences  between  the  mean 
recall  scores  of  the  three  groups,  F(2,68)  «  4.59,  ft  »  .01.  Tukey's  HSD  test 
for  post-hoc  analysis  indicated  that  the  leamers  who  received  the  IV  lesson 
recalled  significantly  more  information  from  the  lesson  than  learners  who 
received  the  TV  lesson.  There  was  no  significant  difference  between  the  mean 
recall  scores  of  the  leamers  who  received  the  ITV  lesson  and  the  mean  recall 
scores  of  the  other  two  groups. 

I\ylental  Effort  and  Inference  Scores 

The  Pearson's  i  test  for  correlation  indicated  thure  was  a  significant 
correlation  between  the  mean  perceived  mental  effort  ratings  and  mean  inference 
scores,  l=.27,  p.  «.012. 

Post-hoc  Analyses 

Several  post-hoc  analyses  were  conducted  in  order  to  explore  possible 
explanations  for  the  results  of  the  primary  analyses. 

Perceived  mental  effort  ratings  and  recall  scores.  The  results  of  the 
Pearson's  i  test  of  correlation  revealed  that  there  was  a  significant  correlation 
between  the  learners*  perceived  mental  effort  ratings  and  their  recall  scores,  t 
=.33,  a  =  .01. 

Lesson  time.  Tho  data  on  lesson  time  were  analyzed  to  determine  if  there 
were  significant  differences  among  treatment  groups  and  to  determine  if  there 
was  a  significant  correlation  between  lesson  time  and  inference  scores  or  recall 
scores.  ANOVA  results  indicated  a  significant  difference  between  the  three 
treatment  groups  in  the  amount  of  time  that  was  required  to  complete  the  lesson, 
£(2,68)  =  8.66,  ii«.0004.  Further  analysis  using  Tukey's  test  revealed  that  the 
IV  (M  =  25.3  minutes)  and  ITV  (M  =  23.2  minutes)  groups  took  significantly 
longer  to  complete  the  lesson  than  the  TV  group  (M  »  18.5  minutes).  However, 
there  was  not  a  significant  correlation  between  reported  lesson  times  and 
learners*  recall  scores,  i  =  .16,  or  inference  sco''3S,  i  =  -0.02. 

Discussion 

The  purpose  of  this  study  was  to  investigate  learners'  preconceptions  of 
the  difficulty  of  interactive  video  (IV),  instructional  television  (ITV),  and 
television  (TV)  and  to  examine  the  effects  of  responding  actively  or  covertly  to 
practice  questions  embedded  in  video-based  materials  on  perceived  mental  effort 
and  achievement  on  a  test  of  recall  and  inference  items. 
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Preconceptions  of  Difficulty 

This  study  was  based  on  the  premise  that  preconceptions  of  a  medium 
influence  the  amount  of  mental  effort  that  learners  invest  in  processing  a 
mediated  lesson.  The  present  study  assessed  learners'  preconceptions  of  the 
difficulty  of  IV.  ITV.  and  TV. 

Although  it  was  predicted  that  adult  learners  would  perceive  IV  as  more 
difficult  than  ITV  and  TV.  and  ITV  as  more  difficult  than  TV.  the  results  were  in 
the  opposite  order.  The  learners  perceived  that  it  was  significantly  easier  to 
learn  from  IV  than  from  ITV  and  TV.  and  significantly  easier  to  learn  from  ITV 
than  from  TV. 

The  differences  between  these  findings  and  the  results  of  Krendrs  (1986) 
and  Salomon's  (1983.  1984-  Salomon  &  Leigh,  1984)  studies  upon  which  the 
predictions  were  based  may  be  due  to  the  differences  in  the  samples  examined  by 
the  researchers.  SalonK)n's  samples  consisted  of  public  school  students  In  grade 
six  and  KrendPs  sample  consisted  of  public  school  students  in  grades  three 
through  ten;  however,  the  participants  In  the  current  study  were  undergraduate 
education  majors  from  a  select  institution  with  stringent  admission 
requirements*  Due  to  the  past  academic  successes  of  the  learners  in  the  sample 
examined  in  the  current  study,  it  is  probable  that  they  had  more  well-developed 
schemata  relative  to  the  learning  process  than  the  majority  of  students  who  are 
enrolled  in  public  elementary  or  secondary  schools. 

Although  the  learners  who  participated  in  the  current  study  perceived 
that  it  was  easier  to  learn  from  interactive  video  than  from  television,  this 
finding  does  not  necessarily  mean  that  the  learners  perceived  interactive  video  as 
'*easy\  College  students  such  as  the  ones  included  in  this  sample  may  be  aware 
that  an  "easy"  lesson  is  not  necessarily  easy  to  "learn  from". 

The  pattern  of  means  on  the  inference  and  recall  test  tended  to  support  the 
learners'  preconceptions  of  the  difficulty  of  learning  from  the  media  utilized  in 
this  study:  Although  the  differences  among  groups  was  not  significant,  the  recall 
and  inference  scores  tended  to  be  higher  for  IV  than  for  ITV  and  TV,  and  higher  for 
ITV  than  for  TV.  In  general,  the  more  difficult  it  was  perceived  to  be  to  learn 
from  a  medium,  the  lower  the  inference  and  recall  scores  of  the  learners  who 
received  a  lesson  using  that  medium. 

These  findings  were  opposite  from  the  results  reported  by  Salomon 
(1983;  1984;  Salomon  &  Leigh.  1984):  Salomon  found  that  learners  scored 
higher  on  a  test  of  inference  when  leaning  from  a  medium  that  was  perceived  as 
difficult  (print)  than  from  a  medium  that  was  perceived  as  easy  (television). 

Although  information  as  to  why  the  learners  perceived  one  medium  as 
easier  or  more  difficult  than  another  was  not  collected,  the  results  indicate  that 
the  learners  did  perceive  IV.  ITV.  and  TV  as  very  different  in  their  ability  to 
facilitate  learning.  Future  research  that  examines  the  learners'  rationale  for 
rating  one  medium  as  easier  or  more  difficult  than  another  may  provide 
information  on  the  extent  to  which  learners  are  aware  of  their  own  cognitive 
strategies  for  learning.  In  addition,  studies  that  are  similar  to  the  current  study 
should  be  conducted  with  other  age  groups  and  aptitude  levels  to  djtermine  to 
what  extent  preconceptions  are  a  function  of  the  age  and  aptitude  of  the  learners. 

Relationship  of  Preconceptions  of  Diffinulty  to  Mental  Effort 

It  was  predicted  that  there  would  be  a  significant  positive  correlation 
between  learners'  preconceptions  of  :!.e  difficulty  of  a  medium  and  their 
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perceived  mental  effort  ratings  when  learning  from  that  medium;  however,  the 
predicated  correlation  did  not  occur. 

The  lack  of  a  significant  correlation  between  preconceptions  of  difficulty 
and  perceived  mental  effort  may  have  been  due  to  the  limitations  of  the 
measurement  instruments.  In  particular,  the  low  reliability  of  the  perceived 
mental  effort  questionnaire  may  have  masked  potential  results. 

The  lack  of  the  predicted  correlation  also  may  have  been  due  to  the 
learners'  interpretation  of  the  questions  on  the  two  questionr<aires.  Whereas  the 
preconceptions  questionnaire  was  designed  to  assess  the  learners'  general 
preconceptions  of  the  difficulty  of  learning  from  a  medium,  the  mental  effort 
questionnaire  was  designed  to  assess  the  amount  of  mental  effort  that  the  learners 
perceived  that  they  had  invested  in  learning  their  specific  assigned  lesson.  The 
preconceptions  questionnaire  required  the  learners  to  rate  the  difficulty  of 
learning  several  tasks  from  each  med'um;  the  mental  effort  questionnaire 
required  the  learners  to  rate  how  hard  they  tried  to  understand  the  lesson,  how 
hard  they  concentrated,  how  hard  the  lesson  was  to  understand,  how  much  the 
lesson  made  them  think,  how  much  they  thought  they  could  remember,  and  how 
much  effort  they  used  in  comprehending  the  lesson.  It  is  possible  the  amount  of 
perceived  effort  required  to  understand,  concentrate,  think,  and  comprehend 
were  not  major  factors  that  learners  considered  as  they  reported  their 
preconceptions  of  the  difficulty  of  learning  from  a  medium. 

Future  researchers  should  use  mental  effort  questionnaires  and 
preconceptions  questionnaires  that  are  closely  aligned  in  wording  in  order  to 
provide  insight  on  the  role  of  preconceptions  in  the  amount  of  mental  effort 
invested  in  processing  a  lesson.  For  example,  the  perceived  mental  effort 
questionnaire  used  in  this  study  asked  the  learners  '^How  hard  did  you  try  to 
understand  the  lesson?";  a  preconceptions  questionnaire  could  be  designed  to 
access  a  similar  construct  by  asking  "How  hard  do  you  usually  try  to  understand  a 
television  lesson?**. 

Perceived  Mental  Effort.  Recall,  and  Inference 

It  was  expected  that  altering  the  typical  characteristics  of  video-based 
materials  through  the  addition  of  practice  questions  requiring  an  overt  (IV)  or 
covert  (ITV)  response  would  overcome  learners'  preconceptions  of  TV  as  a 
passive  medium  requiring  little  mental  effort;  therefore,  they  would  engage  in 
actively  processing  the  content,  perceive  that  they  were  investing  a  large  amount 
of  mental  effort,  and  achieve  high  scores  on  a  iest  of  inference  items. 

Although  the  manipulation  of  the  characteristics  of  the  medium  did  not 
affect  the  learners'  perceived  mental  effort  ratings  or  inference  scores,  it  had  a 
significant  effect  on  the  learners'  recall  scores.  The  data  analyses  indicated  that 
the  learners  who  received  the  IV  lesson  that  required  an  active  response  recalled 
significantly  more  information  from  the  lesson  on  a  test  of  recall  than  learners 
who  received  the  TV  lesson  that  did  not  require  a  response.  Actively  responding 
to  the  practice  questions  in  the  IV  lesson  did  not  result  in  significantly  greater 
recall  scores  than  covertly  responding  to  the  questions  in  the  ITV  lesson,  and 
covertly  responding  to  the  practice  questions  in  the  ITV  lesson  did  not  result  in 
significantly  greater  recall  scores  than  viewing  the  TV  lesson  without  practice 
questions. 

These  results  were  not  consistent  with  the  predictions,  but  upon  closer 
examination  of  the  learning  task,  the  results  do  not  seem  surprising.  In  the 
current  study,  the  learners  received  practice  questions  at  the  verbal  recall 
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level,  hence  they  were  provided  with  direct  practice  toward  the  recall  posttest. 
The  practice  questions  were  designed  to  ensure  that  the  learners  possessed  the 
factual  knowledge  that  was  necessary  for  the  generation  of  inferences.  The 
learners  were  expected  to  connect  the  individual  facts  to  create  new  inferences; 
however,  they  did  not  receive  practice  in  the  connection  of  the  facts. 

Several  researchers  (Heestand,  1985;  Uetke  &  Gall,  1985)  have  found 
that  factual  postquestions  facilitated  the  recall  of  the  practiced  information,  but 
had  no  significant  effect  on  the  recall  of  information  that  was  not  practiced. 
Other  researchers  (Dalton  &  Hannafin,  1987;  Lietke  &  Gall,  1985)  have 
suggested  that  it  is  difficult  to  teach  higher-level  cognitive  skills  through  the  use 
of  lower-level  questioning  strategies.  It  is  possible  that  the  recall  practice 
questions  may  have  caused  the  learners  to  focus  on  the  rehearsal  of  the  verbal 
information  rather  than  on  the  elaboration  of  the  content. 

The  fact  that  learners  in  the  IV  group  reported  significantly  longer  lesson 
times  than  the  learners  in  the  TV  group  may  be  offered  as  an  alternative 
hypothesis  to  explain  the  results.  But  although  there  was  a  significant  difference 
among  the  three  treatments  in  the  time  spent  on  the  lesson,  there  was  not  a 
significant  correlation  between  reported  lesson-time  and  recall  scores.  These 
results  suggest  that  the  differences  in  the  recall  scores  of  the  IV  group  and  the  TV 
group  were  not  entirely  due  to  the  greater  amount  of  time  spent  on  the  lesson  by 
the  learners  in  the  IV  group. 

The  results  of  this  study  indicate  that  learners  who  are  required  to 
actively  respond  to  practice  questions  that  are  embedded  in  a  video-based  lesson 
recall  significantly  more  information  from  a  video-based  lesson  than  learners 
who  are  not  provided  with  practice  questions.  The  results  also  indicate  that  the 
inclusion  of  practice  questions  at  the  verbal  recall  level  may  not  result  in 
significant  increases  in  the  inferences  generated  in  response  to  a  video-based 
lesson. 

However,  the  compressed  range  of  the  inference  scores  may  have  masked 
potential  differences  among  the  treatment  groups.  Out  of  18  possible  inferences, 
the  mean  of  the  IV  group  was  5.04,  the  mean  of  the  ITV  group  was  4.60,  and  the 
mean  of  the  TV  group  was  4.47.  The  type  of  practice  question  utilized  in  this 
study  may  hRvc  inhibited  the  generation  of  inferences,  however,  the  compressed 
range  of  scores  may  also  have  been  due  to  the  difficulty  of  the  scientific  content 
for  the  non-science  majors  that  participaled  in  the  current  study. 

Future  researchers  should  continue  to  investigate  the  effects  of  practice 
questions  with  other  types  of  practice  questions,  other  types  of  objectives,  and 
other  groups  of  learners.  Research  that  incorporates  recall  test  items  that  are 
not  directly  matched  to  the  practice  questions  may  provide  valuable  insight  as  to 
whether  increased  recall  scores  are  influenced  by  the  direct  rehearsal  of  the 
infonnation  that  is  included  on  the  posttest  or  by  a  general  review  of  the 
information  that  precedes  the  question.  The  effect  of  practice  questions  at 
higher-levels  of  cognitive  processing  seems  tc  be  an  especially  promising  area 
for  further  research.  It  is  possible  that  providing  learners  with  practice 
questions  at  the  inference  level  would  result  in  the  generation  of  a  larger  number 
of  inferences  than  was  obsen/ed  in  the  current  study.  The  effect  of  practice 
questions  on  the  achievement  of  higher  levels  of  learning  ouiuomes  such  as 
intellectual  skills  or  applications  also  seems  to  be  a  promising  area  for  further 
research.  Research  with  learners  of  different  ages,  aptitudes,  and  motivational 
levels  than  were  used  in  the  current  study  may  strengthen  the  findings  and 
increase  the  generalizability  of  the  results. 
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Consistent  with  the  predictions,  a  modest  significant  correlation  was 
found  t)etween  the  learners'  mean  inference  scores  and  mean  perceived  mental 
effort  ratings*  The  post-hoc  analyses  indicated  there  was  also  a  modest 
significant  correlation  between  the  learners*  perceived  mental  effort  ratings  and 
their  recall  scores.  As  learners*  perceived  mental  effort  increased,  their  recall 
and  inference  scores  also  increased* 

But  although  it  was  predicted  that  learners  would  perceive  that  they 
invested  more  mental  effort  in  processing  the  IV  lesson  thjn  in  processing  the  ITV 
lesson  and  TV  lesson,  and  that  learners  would  perceive  that  they  invested  more 
mental  effort  in  processing  an  ITV  lesson  than  in  processing  a  TV  lesson,  there 
v/as  no  significant  difference  between  the  three  groups. 

The  prediction  was  based  on  l>e  results  of  research  studies  (Britton, 
Piha,  Davis,  &  Wehausen,  1978;  Burton,  Niles  &  Lalik,  1986;  Reynolds  & 
Anderson,  1982)  that  found  the  amount  of  cognitive  capacity  usage,  as  measured 
by  a  secondary  task  technique,  increased  when  practk)e  questions  were  embedded 
in  text-based  lessons.  Although  cognitive  capacity  usage  and  mental  effort  have 
been  operationally  defined  in  a  similar  manner  by  Salomon  (1981),  it  Is 
possible  that  the  self-report  measure  used  in  the  current  study  was  not  as 
sensitive  to  increases  in  mental  processing  as  the  secondary  task  measures  that 
were  used  in  the  text-based  studies,  and  in  fact,  may  not  have  actually  assessed 
the  same  cognitive  processes. 

The  low  reliability  of  the  perceived  mental  effort  questionnaire 
(Cronbach's  alpha  «  .55)  indicates  that  future  researchers  should  use  a  different 
method  of  assessing  perceived  mental  effort.  The  use  of  secondary  task  technique 
to  assess  cognitive  capacity  usage  when  learning  from  video-based  materials  may 
help  determine  if  the  inclusion  of  practice  questions  affects  the  amount  of  mental 
effort  invested  in  processing  video-based  instruction  and  text-based  instruction 
in  a  similar  manner. 

Conclusions 

The  conclusions  of  this  study  support  the  current  practice  of  including 
practice  qnestions  requiring  an  active  response  in  the  design  of  interactive  video 
instructional  materials.  The  results  suggest  that  responding  actively  to  practice 
questions  at  the  verbal  recall  level  will  increase  the  amount  of  information 
recalled  from  a  video-based  lesson  that  is  designed  to  teach  factual  information  at 
the  verbal  recall  level.  However,  it  appears  that  responding  actively  to  practice 
questions  at  the  verbal  recall  level  will  not  increase  the  number  of  inferences 
generated  in  response  to  a  video-based  lesson.  These  findings  suggest  that  video- 
based  lessons  designed  to  teach  more  than  simple  recall  may  need  to  include 
higher-level  practice  questions  in  order  to  increase  the  amount  of  information 
learned  from  the  lesson.  The  results  suggest  that  additional  research  is  needed  on 
the  effects  of  actively  and  covertly  responding  to  practice  questions  in  video- 
based  lessons. 

Additional  research  also  is  needed  on  ways  that  different  media  are 
perceived.  It  has  been  suggested  that  preconceptions  of  media  may  vary  depending 
in  the  age  and  aptitude  of  the  learner. 

Further  research  that  examines  learners*  preconceptions  of  media  and  the 
ways  that  preconceptions  can  be  altered  to  result  in  increased  mental  effort  may 
be  able  to  provide  valuable  information  that  may  result  in  more  effectively 
designed  video-based  lessons. 
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Foixrth  Generation  Instructional  Design  Model: 
An  Elaboration  on  Authoring  Activities 


Instructional  Design 

Instruction  is  a  process  that  involves  both  the  planning  of  an  environment  where 
learning  can  take  place  and  the  delivery  of  information  to  the  learner.  How  to 
improve  learning  in  a  planned  envu-onment  has  been  a  major  concern  of 
educational  psychologists  since  the  first  formal  research  on  classroom  learning  by 
Edward  L.  Thomdike  (1969).  Since  Thomdike,  learning  in  the  classroom  (or 
training)  environment  has  been  a  focus  of  educational  psychologists  who,  unlike 
experimental  psychologists  who  have  limited  their  study  of  learning  to  the 
laboratory,  consider  the  complexity  of  factors  that  influence  the  effectiveness  and 
efficiency  of  classroom  instruction  on  improving  learning. 

Factors  that  contribute  to  the  divergence  in  student  learning  include  cognitive  and 
affective  variables,  student  knowledge,  stimulus  properties,  delivery  system  (e.g., 
lecture,  computer,  video,  etc.),  management  procedures  (e.g.,  program  control, 
learner  control),  instructional  strategies,  and  evaluation  methods. 

Although  educational  psychologists,  as  well  as  other  educators,  have  long  dealt  with 
these  factors,  it  has  only  been  in  the  past  decade  that  tb.ey  have  investigated  variables 
and  conditions  directly  related  to  the  planning  of  a  total  learning  environment;  that 
is,  recognizing  the  interaction  of  multiple  factors  that  go  into  designing  and 
implementing  instruction.  Instructional  systems  design  (ISO)  is  the  term  applied  to 
this  planning  effort;  it  includes  the  processes  of  development,  evaluation,  and 
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management.  The  IGD  concept  of  development  uses  a  scientific  process  that 
involves  both  the  planning  of  an  environment  where  learning  can  take  place  and 
the  delivery  of  information  to  the  learner. 
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Growth  of  this  approach  to  instructional  desigix  has  been  facilitated  by  rr. r  inputs 
from  fields  of  inquiry  outside  of  education,  such  as  computer  science,  management 
information  sciences,  and  psychology;  and  within  the  field  especially  from 
instructional  technology.  Whenever  possible,  research  and  theory  from  these  areas 
have  been  applied  to  appropriate  phases  in  the  upd^.ted  !D  model.  Of  these  areas, 
computer  science  has  had  a  major  impact  with  such  contributions  as  system  design 
strategy  variables  (e.g.,  system  architecture  and  software  developments),  structure  of 
information  (both  as  external  information  and  internal  representations),  efficiency 
of  design  concepts  (e.g.,  heiu-istic  methods),  and  alternative  learner  control  methods. 

In  the  updated  ID  model  the  emphasize  is  on  the  basic  principles  of  design  using  an 
iterative  approach  to  development  rather  than  a  conventional  algorithmic  process 
model  in  which  the  output  of  one  step  becomes  the  input  for  the  next.  This  is  done 
for  two  reasons.  First,  the  complexity  of  the  ID  process  is  a  result  of  alternatives 
available  for  solving  domain-  specific  learrung  problems.  Thus,  within  phases  the 
author  has  numerous  options  to  be  organized  to  fit  the  given  situation.  Second,  the 
authoring  activities  can  bt  '"idependent  of  the  decision  making.  Thus,  in  the  design 
of  an  expert  instructional  desig  system,  the  knowledge  base  can  be  separate  from  an 
executive  <!*^^ategy  system. 

The  principles  of  design  in  the  updated  ID  model  are  based  primarily  upon  learning 
and  instructional  research  and  theory  and,  just  as  importantly,  upon  applied 
development  experience.   It  is  one  thing  to  talk  and  write  about  the  instructional 
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design  process,  and  quite  another  to  have  actually  done  it.  As  a  design  science  field, 
much  like  engineering.  Instructional  design  relies  upon  field  experience  for  much  of 
the  "how  to  do  it"  components  of  its  system.  In  this  regard,  the  conglomerated 
authoring  activities  exhibit  different  degrees  of  theory-base  and  empirical  base.  For 
example,  the  analysis  draws  heavily  on  learning  theory  to  set  the  conditions  of 
learning  and  evaluation,  m  contrast,  the  development  draws  on  field  experience  to 
define  activities. 


Another  important  trend  toward  a  scientific  approach  to  instructional  design  is  the 
increased  emphasis  placed  on  the  use  of  evaluation  (Tennyson,  1978;  also  Tennyson 
paper  AECT  1990).  Although  evaluation  has  always  been  part  of  the  instructional 
design  plan,  it  traditionally  was  relegated  to  the  last  step  in  the  process.  Even  the 
most  recent  models  (e.g.,  Merrill,  1983;  Reigeluth,  1983)  have  only  a  minimal 
consideration  of  evaluation.  The  impetus  for  increased  use  of  evaluation  in 
development  is  part  of  a  larger  societal  mo^pment  towards  accountability  in 
education  and  training.  Following  the  guidelines  presented  by  Tennyson's  theory- 
based  model  (Tennyson,  1990\  evaluation  has  become  an  integral  part  of  the  design 
system. 

The  various  facets  and  combinations  of  the  updated  ID  process  (4th  generation)- 
Analysis,  Analysis-Design,  Analysis-Maintenance,  Design,  Design-Development, 
Design-Development-.mplement,  Development-Implement,  Development- 
Implement-Maintenance,  Implement,  Implement-Maintenance  and  Maintenance 
will  be  presented  in  terms  of  the  authoring  activities  associated  with  each  facet  of  the 
updated  ID  model.   Figure  one  presents  the  updated  ID  model  (fourth  generation)  as 
a  dynamic  processing  model  in  referenr^  to  the  elaboration  of  authoring  activities 
(for  further  detail  on  updated  ID  model  see  Tennyson's  presentation  and  paper). 
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Figure  1.  4th  Generation  ISD  Model. 


e  1S90  Ttnnyton  &  Chritt«nt«n  (1990) 
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Analysis 


Authoring  Activities 


Define  philosophy  and  theory 
of  learning  conditions 


Analyze  target  population 


Feasibility  evaluation 


Analy7e  document 


These  conditions  influence  each 
step  in  the  ID  process  Thus, 
the  generic  steps  are  adjusted 
to  account  for  the  defined 
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ability, 
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Determine  learner  differences: 

cognitive  style, 

aptitude, 

learning  style, 
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perception 


Validate  the  analysis  process 
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learning  environment 
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Analysis-Design 

Define  learning  variables 
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learned 


Define  the  learning  environment 
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Analysis-Design 

Specify  goals 

Specify  learning  objectives 


Authoring  Activities 

State  abstract  descriptions  of  what 
knowledge  is  to  be  acquired  (levels  of 
knowledge — declarative,  conceptual, 
and  procediu-al) 


State  objectives  for  learning  program, 
specifying: 

desired  conditions  of  learning  (e.g. 
verbal  information,  intellectual 
skills,  cognitive  strategies,  motor 
skills,  attitudes) 


Define  management  and  delivery 
system 

Establish  role  of  computer  in 
mar  gement  of  the  learning 
environment  Identify  basic  goals  for 
computer  delivery  of  instruction  (and 
identify  other  alternative  systems; 
e.g.,  interactive  video) 

Define  specifications  of  instruction 

Document  conditions  and 
specifications  of  program: 
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allowable  media, 
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Authoring  Activities 


Plan  design  and  development 
effort 

Consider  whether  to: 

buy  and  use  existing  materials, 

modify  an  existing  course,  develop  a 

new  course,  or  discontinue 

development  effort  Estimate  costs 

and  resource  requirements  for  each 

alternative 

Analyze  information  (micro) 

Knowledge  engineering  activities: 
Identify  organization  of  information 
Establish  knowledge  base 
Determine  schematic  structure  from 
knowledge  base  (referenced  to 
conditions  of  learning)  Determine 
schematic  structure  from  semantic 
structure    Determine  semantic 
structure  for  content  attribute 
characteristics 

Analysis-Maintenance 

Analyze  learning  needs  and/ 
or  problem 

Identify  discrepancies  between 
desired  and  actual  learning  and 
performance  Determine 
consequences  of  learning  and 
performance  discrepancies 

Define  constraints  restricting 
resolution  of  learning  and 
performance  discrepancies. 

Identify  the  scope  of  the 

need /problem  (i.e.,  curriculum, 

course,  module  and /or  lesson) 

Maintenance  Evaluation 
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Design 


Define  entry  knowledge 


Define  organization  and 
sequence  of  information 


Specify  formative  evaluation 
system 


Prepare  design  document 


Design-Development 

Specify  instructional  system 


Specify  meta-instructional 
strategies 


Authoring  Activities 


Identify  and  determine 
learner  entry  knowledge  and 
behaviors  Determine 
learner  (student)  model: 
backgroimd  knov/ledge, 
associative  knowledge, 
prerequisite  knowledge, 
prior  knowledge 


Determine  sequence  of 
information  through: 

a)  course, 
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Document  all  design 
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Design-Development 


Authoring  Activities 


Spedfy  mode  of  interaction 


Specify  screen  management 


Specify  presentation  modes 


Specify  computer-based 
enhancements 


Specify  methods  of  management 


definition,  context,  best 
examples,  expository 
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initiative 
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level  of  intelligence: 
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Spedfy  message  design 


Select  display  diaracteristics 
(e.g.,  graphics,  text,  color) 
Design  screen  layout 
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Design-Development 


Authoring  Activities 


Specify  human  factors 

Design: 

menus,  function  key 
prompts,  special  helps 
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configurations 

Review/select  existing  materials 

Select  portions  of  existing 
materials  appropriate  for 
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Define  situational  variables 

Identify  existing  materials, 
compare  them  with 
needs/problem  Identify 
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Design-Development-  Implement 

Define  conditions  of  learner 
assessment/evaluation 

Determine  the  method(s)  for 
assessing  and  evaluating 
learner  knowledge 
acquisition  (e.g.,  methods  of 
diagnosis,  error  detection, 
error  analysis) 


Specify  learner  evaluation  system 

Determine  on-task  learning 
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Development  Authoring  Activities 

Prepare  content  narratives 

Acquire  and  document 
subject  matter  content  (i.e., 
knowledge  base  and 
schematic  structure) 

Prepare  learning  activity 
designs 

Review  learning  activity 
designs  and  associated 
content  for  adherence  to 
design  and  for  acciu-acy  and 
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Develop  learning  activities 
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instructional  strategies 

Development-Implement 

Editing  of  learning  program 

Establish  format  and 
composition  requirements 
Review  all  materials  for 
grammar,  style  and 
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Formative  evaluation 

Conduct  one-on-one  tryout 
of  prototype  materials 
Revise  on  the  basis  of  one- 
on-one  result  Conduct 
simulation  tryout  Refine  on 
the  basis  of  simulation  test 
Edit  and  produce  Perform 
technical  and  mechanical 
review 

Documentation 
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Design  Implement-Maintenance 

Develop  assessment  instrument 


Reviews 


Authoring  Activities 


Develop  items  appropriate 
for  each  objective  and 
learning  activity  Develop 
items  consistent  with 
designed  assessment  system 


Subject  matter  experts 
review  material  for  accuracy 
and  completeness  Designers 
review  material  to 
determine  whether  it  meets 
requirements  established  in 
analysis  and  development 
phases 


Implement 

Reproduce  materials 
Establish/modify  support 
services 

Distribute  materials 
Deliver  instruction 
Collect  data  on  learner 
performance  and  learner 
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Implement-Maintenance 

Summative  Evaluation 


Analyze  data 
Distribute  report  and 
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Determine  whether  to  make 
major  revision  (go  to 
Analysis)  or  minor 
maintenance  (go  to 
Maintenance) 
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Maintenance  Authoring  Activities 


Perform  maintenance  on 
learning  activities  and  test 
items. 
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In  recent  years,  researchers  in  our  field  have  been  engrossed  in  ongoing 
arguments  about  the  value  of  research  with  media,  research  on  media,  and 
research  on  learning  from  media.  Richard  Clark,  for  example,  has  expended 
untold  hours  explaining  the  effects  created  by  the  gap  between  descriptive 
theory  and  applied  research  (Clark  &  Snow,  1975).  Among  other  claims 
(Clark,  1983),  Clark  has  concluded  that  .  .  media  attributes  are 
interchangeable  and  make  no  necessary  psychological  contribution  to 
learning"  (Clark,  1987).  Based  upon  the  available  evidence,  this,  of  course,  is 
true.  However,  what  Richard  Clark  did  not  tell  us  is  that  media  attributes 
may  make  a  sociological  and/or  physiological  contribution  to  learning.  He 
did  not  tell  us  this  because,  for  many  reasons,  we  did  not  ask  the  question.  As 
a  result,  one  is  led  to  the  conclusion  that  our  narrowly  focused  questions  may 
be  a  cause  for  concern  (Clark,  1988). 

Last  year,  Clark  (1989)  concluded  that .  .  research  quantity  indicators  do  not 
support  quality  judgments."  He  went  on  to  say  that  two  major  changes  need 
to  occur  in  the  way  that  research  is  conducted  in  the  future.  First,  .  .  more 
effort  needs  to  be  spent  at  the  *front  end*  or  conceptualization  stage  of 
studies"  and  second,  .  .  our  studies  need  to  stop  emphasizing  descriptive 
research  and  instead  focus  on  prescriptive  research  designs  and  questions." 

My  purpose  is  threefold.  One  is  to  reiterate  the  distinctions  between 
disciplines  and  applied  fields  of  study  and  suggest  that  these  distinctions 
should  be  evident  in  the  descriptive/ prescriptive  research  reports  generated 
from  each.  A  second  is  to  impart  my  belief  that  replicating  the  theoretical  and 
inquiry  strategies  of  descriptive  research  may  be  debilitating  applied  research. 
A  third  is  to  discuss  instruction  in  reality  and  the  implications  of  a  holistic 
research  paradigm  to  study  learning  from  instruction  (The  paradigm  even 
includes  the  teacher/designer  as  an  essential  variable). 
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DISCIPLINES  AND  APPUED  HELDS 

Education  is  an  applied  field  of  study.  Therefore,  if  education  is  an  applied 
field  of  study,  like  medicine  and  law  (not  a  discipline,  like  mathematics  an  i 
chemistry),  then  so  are  the  knowledge  components  that  comprise  it: 
instruction,  curriculum,  counseling,  evaluation,  and  administration 
(Beauchamp,  1968). 

In  the  classical  sense,  a  discipline  of  knowledge  includes  research  based  upon 
mature  theories  that  will  encompass  and/or  correspond  with  both  existing 
and  new  microtheories  and  m\cro theories;  the  general  concern  is  with 
delineating  mature  theories  that  must  specify  the  dynamic  causal 
relationships  that  exist  within,  between,  and  among  all  of  the  elements 
comprising  the  domain.  Traditionally,  the  goal  of  basic  research  have  been  to 
generate  new  descriptive  knowledge,  if,  for  no  other  reason,  the  sake  of 
knowing. 

Genf^rally,  an  applied  field  of  study  includes  research  based  upon  knowledge 
gleaned  from  one  or  more  disciplines;  the  concern  is  with  identifying  eclectic 
systems,  networks,  paradigms,  et  cetera,  to  infer  the  ideal  structure  of 
functional,  reality  ladened,  knowledge.  The  goal  of  applied  research,  in  most 
cases,  has  been  to  generate  new  knowledge  about  designing,  developing, 
implementing,  managing,  and/or  evaluating  the  effective  (natural  science) 
and/or  affective  (social  science)  elements  within  the  universe. 

Therefore,  the  task  of  researchers  and  other  professionals  within  an  applied 
field  of  study  is  to  translate  and  transform  the  "knowing  what"  descriptive 
knowledge  generated  in  the  disciplines  to  the  "knowing  how"  prescriptive 
knowledge  so  necessary  in  an  applied  field  of  study.  One  must  remember  that 
the  major  difference  between  basic  descriptive  research  and  applied 
prescriptive  research  is  reality.  I  mention  the  distinction  only  because  it  is  so 
easy  to  lose  sight  of  our  purpose. 

THE  PROBLEM  OF  PURPOSE 

It  is  my  belief  that  at  least  part  of  the  "purpose"  problem,  if  it  is  generally 
conceived  to  be  a  problem,  stems  from  the  fact  that  many  of  the  graduate 
programs  of  education  (an  applied  field  of  study)  have  mirrored  the 
theoretical  and  methodological  inquiry  strategies  and  techniques  of  science  (a 
discipline).  It  follows,  then,  that  our  orientations  about  the  theoretical 
constructs  of  research,  in  general,  and  methodology,  in  particular,  have 
reflected  and  evolved  from  the  "laboratory  setting,"  where  conditions  and 
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methods  can  usually  be  controlled,  through  statistics  and /or  randomization, 
to  ignore  potential  dynamic  causal  relationships  that  exist  within  the  domain 
of  reality,  but  outside  the  domain  of  immediate  concern  or  interest  to  the 
researcher.  In  reality,  I  am  not  implying  that  this  model  or  the  descriptive 
data  that  result  from  research  conducted  in  a  laboratory  setting  are  good  or 
bad;  that  question  cannot  be  answered  until  we  have  incorporated  and 
evaluated  these  data  in  our  own  applied  research,  in  reality.  Some  would 
argue  that  this  is  an  extension  of  the  controversy  between  reductionists  and 
naturalists  (Guba,  1981;  Magoon,  1977),  while  others  will  see  it  as  a  distinction 
between  the  professional  research  activities  of  those  in  a  discipline  and  those 
in  an  applied  field  of  study. 

To  summarize,  the  task  of  synthesizing  desaiptive  data  i'rom  the  **root  social 
sciences"  (Clark,  1989)  to  form  a  usable  paradigm  for  applied  research  is  quite 
different  than  the  task  of  employing,  verbatim,  the  theoretical  and 
methodological  constructs  of  those  same  sciences.  Certainly  learner 
competence  is  a  gOt>l  or  end  product  that  applied  fields  concerned  with 
instructional  theory  hold  in  common  with  the  root  social  sciences  concerned 
with  learning/ cognitive  theory,  but  the  means  of  arriving  at  the  end  product 
is,  or  should  be,  different. 

In  the  past,  our  commitment  to  the  constructs  imposed  by  a  particular 
discipline  has  failed  to  produce  the  consistency,  generality,  and  commonality 
needed  for  the  development  of  a  comprehensive  paradigm  for  applied 
research  on  learning  from  instruction.  Frustration  with  the  limited  yields 
from  past  research  has  prompted  calls  for  new  research  directions.  However, 
research  without  a  theoretical  domain  may  provide  new  directions  which  are 
no  more  beneficial  as  a  guide  to  studying  learning  from  instruction  than  the 
paths  of  the  past.  In  the  future,  the  relationship  between  learning/cognitive 
theory  research  and  learning  from  instruction  research  must  be  reciprocal; 
each  should  serve,  in  part,  to  strengthen  the  other.  Without  some 
ackno>v^ledgement  of  the  effective  and  affective  processes  involved  in 
learning  from  instruction,  applied  research  efforts  will  continue  to  produce 
extremely  limited  observations.  Learning  from  instruction  should  be  based, 
in  part,  upon  the  constructs  of  learning/ cognitive  theory,  but  instructional 
theory  should  not  be  confused  with  learning/cognitive  theory. 

INSTRUCTION  IN  REALITY 

Perhaps  the  most  pervasive  problem  with  research  on  learning  from 
instruction  has  been  the  lack  of  inclusiveness;  our  inattention  to  the  totality 
of  the  instructional  environment.  Separately,  we  have  covered  several 
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variables,  usually  more  than  one  at  a  time,  including  learners,  teachers, 
treatment  characteristics,  environmental  or  situational  conditions,  intrinsic 
and  extrinsic  motivation,  media  characteristics,  and  factors  related  to  the 
instructional  message  or  task.  Research  has  been  conducted  from  the 
individual  researcher's  conception  of  fundamental  combinations  of  pertinent 
variables.  Based  upon  what  we  have  learned  from  our  past  research,  it  seems 
that  one  of  the  shortcomings  has  been  our  failure  to  consider  certain  variables 
and  relationships  that  have  been  treated  and  evaluated  by  our  colleagues  in 
the  disciplines  and  other  applied  fields.  To  illustrate,  the  learning /cognitive 
theorist  does  not  focus  on  the  specific  elements  of  one  essential  component 
within  a  learnin;5/ cognitive  model  at  the  expense  of  the  others.  For,  by 
themselves,  the  essential  components  are  not  theory:  even  though  the 
within  component  elements  play  an  integral  part  in  defining  a  theoretical 
domain,  in  the  final  analysis,  they  often  become  subordinate  to  the 
relationships  between  and  among  component  elements.  This  should  also  be 
the  case  with  paradigms  to  study  learning  from  instruction. 

A  PARADIGM  TO  STUDY  LEARNING  FROM  INSTRUCTION 

Although  the  paradigm  described  here  has  implications  for  instructional 
design,  it  is  neither  an  instructional  design  paradigm  nor  a  paradigm  of 
instruction;  rather,  it  is  a  research  paradigm  to  study  learning  from 
instruction.  The  paradigm  is  based  upon  the  presumption  that  there  is  both 
theoretical  and  intuitive  commonality  among  the  researchers  and 
practitioners  interested  in  learning  from  instruction.  Therefore,  for  such  a 
paradigm  to  be  of  value  to  all  researchers  vdthin  our  applied  field  of  study,  it 
must  be  eclectic.  It  must  be  broad  enough  to  include  our  psychological, 
sociological,  and  physiological  experimental  and  naturalistic  research 
interests,  and  dynamic  enough  to  allow  for  the  inclusion  of  various  theories 
pertaining  to  how,  when,  where,  and  why  learners  become  competent. 

The  clinical  paradigm  proposed  here  to  study  learning  from  instruction  is 
based  upon  five  essential  variables  of  instruction:  the  learner,  the 
teacher/designer,  the  task,  the  resources,  and  the  instructional  environment. 
It  follows,  then,  that  if  learner  competence  is  the  ultimate  product  of  the 
instructional  environment,  then  an  effective  and  affective  instructional 
environment  is  the  product  of  the  interactions  from  within,  between,  and 
among  the  teacher/designer,  the  learner,  the  task,  and  the  resources.  The 
psychological,  sociological,  and  physiological  dimensions  of  the  variables  of 
instruction  are  used  to  further  describe  the  distinctiveness  of  the  static  and 
dynamic  attributes  innately  existing  within,  or  designed  into,  each  of  the 
essential  variables.  Even  though  the  proposed  dimensions  may  not  be 
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mutually  exclusive,  this  deficiency  is  minimized  by  viewing  learning  from 
instruction  as  a  dynamic  process.  However,  this  should  not  be  construed  to 
mean  that  the  dynamic  and  static  attributes  existing  within,  or  designed  into, 
the  psychological,  sociological,  and  physiological  dimensions  of  the  essential 
variables  of  instruction  are  not  important  indicators  of  potential  interactions. 
They  are,  in  fact,  of  great  import  to  our  efforts  in  diagnosing  and  prescribing 
the  between  and  among  relationships  of  the  identified  attributes  associated 
with  each  of  the  variables  of  instruction. 

On  the  one  hand,  learner  competence  is  contingent  upon  the  ability  of  the 
motivated  learner  to  select,  amplify,  and/or  manipulate  the  attributes  of  the 
other  variables  of  instruction  to  form  an  environment  that  is  qualitatively 
distinct.  On  the  other  hand,  learner  competence  is  contingent  upon  the 
ability  of  the  teacher/designer  to  select,  amplify,  and/or  manipulate  the 
atnibutes  of  the  other  variables  of  instruction  to  form  an  environn?ent  that  is 
quantitatively  distinct.  The  implication  of  such  within,  between,  and  among 
dependence  of  the  learner  and  the  t€  cher /designer  is  that  causal 
relationships  within  the  instructional  environment  result  from  the  abilities 
of  the  learner  to  integrate  the  functional  relationships  between  and  among 
the  other  variables  in  a  schema  that  will  result  in  a  theoretically  predictable 
performance.  Functional  relationships  of  the  dynamic  and  static  attributes 
within  each  variable  and  causal  relationships  between  and  among  the 
variables  of  instruction  and  learner  competence  are  best  predicted  from 
knowledge  about  the  intermediary  processes  that  occur  within  the  learner  as 
well  as  the  potential  effective  and/or  affective  relationships  that  occur 
between  and  among  all  of  the  other  variables  of  instruction.  Hence,  the 
objective  form  and  subjective  meaning  of  the  learning  task  must  yield  both 
functional  and  causal  distinctions  in  terms  of  the  cognitive,  affective,  and/or 
psychomotor  requirements  of  the  learner. 

Research  proceeding  from  this  conceptual  level  requires  not  only  the 
examination  of  the  essential  variables  of  instruction  (teacher/designer, 
learner,  task,  resources,  and  the  instructional  environment)  and  their 
attributes  (psychological,  sociological,  and  physiological),  but  of  the  essential 
intermediary  processes  (functional  and  causal  relationships)  as  well. 
Together,  these  essential  variables  can  serve  as  common  denominators  for  a 
variety  of  research  interests  in  applied  fields  such  as  instructional  technology. 
Moreover,  theoretical  integrity  would  be  established  and /or  advanced  if  these 
components  were  either  implicit  or  explicit  features  of  most,  if  not  all, 
research  paradigms. 
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ABSTRACT 


In  the  past  century,  much  research  has  addressed  the  legibility 
of  text  in  print,  Sanford  (1888),  Griffing  and  Franz  (1896)  and 
Tinker  (1963)  have  contributed  to  this  body  of  knowledge.  Studies 
were  performed  to  determine  the  best  length  of  line,  color,  and 
line  spacing,  etc.  More  recently,  researchers  have  found  that 
reading  rate  is  adversely  affected  by  the  legibility  of  the 
electronic  screen  text.  Most  legibility  studies  used  adult 
subjects. 

The  only  reading  from  a  computer  screen  studies  involving  junior 
high  school  students  were  by  Feldmann  and  Fish  (1987,  1988).  The 
ma j  or  focus  of  the  study ,  was  to  determine  i f  reading  from  the 
computer  screen  affected  junior  high  school  students' 
comprehension.  Additionally,  the  study  questioned  whether  reading 
rates  were  affected  by  the  legibility  of  electronic  screen  text. 


The  present  study  investigated  differences  in  reading  rate  and 
comprehension  scores  of  eighth  grade  students  on  a  cloze  reading 
exercise  involving  the  reading  of  a  cloze  passage  from  ten 
different  modes  text  of  presentation.  The  study  was  designed  to 
compare  reading  rate  and  comprehension  scores  of  junior  high 
students  reading  from  paper  copy  or  computer  screen.  In  addition, 
the  study  was  designed  to  determine  optimal  electronic  screen  text 
presentation  modes,  involving  line  length,  background  and 
foreground  color,  and  contrast  ratio. 

Two  one-way  ANCOVAs  were  used  to  analyze  the  data.  One  ANCOVA 
analyzed  reading  rate  while  the  other  analyzed  the  comprehension 
scores.  The  ANCOVA  results  indicated  that  there  were  no 
statistically  significant  differences  in  reading  rate  or 
comprehension  score  for  any  of  the  ten  groups. 

The  findings  of  no  differences  in  reading  speed  from  electronic 
screens  to  paper  conflicts  with  most  of  the  findings  on  adult 
reading  rate  research  (Gould  &  Grischkowsky ,  1984,  1986;  Hathaway, 
1984) .  Reasons  accounting  for  the  difference  in  findings  may  be: 
(a)  the  reading  of  the  adult  subjects  was  superior  to  the  eighth 
grade  students;  (b)  the  nature  of  the  reading  materials, 
proofreading  as  opposed  to  a  cloze  passage;  and  (c)  the  focus  on 
groups  of  students  rather  than  the  individualized  treatment  of 
adults.  This  would  suggest  that  reading  level  might  be  a  factor 
affecting  reading  rate  from  electronic  screens  or  paper. 

The  finding  of  no  difference  in  comprehension  was  consistent 
with  previous  studies.  Thus,  the  ability  of  adults  or  eighth  grade 
students,  to  comprehend  text  does  not  appear  to  be  impaired  by  the 
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decreased  legibility  of  computer  screens.  Although  the  reading 
rate  of  adults  seems  to  be  effected  by  the  legibility  of  computer 
screens,  eighth  grade  students*  reading  rates  do  not  appear  to  be 
adversely  affected. 
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Paper  versus  CRT  -  Are  Reading  Rate  and 
Comprehension  Affected? 


Computers  and  electronic  screens  have  become  integral  parts 
of  American  society.  Office  workers,  engineers,  scientists, 
designers,  writers,  accountants,  and  many  others  spend  long  hours 
in  front  of  electronic  screens.  Often,  the  amount  of  text 
presented  on  the  screen  is  considerable. 

Student  computer  usage  in  schools  has  increased  rapidly. 
There  are  approximately  52  thousand  computers,  for  instructional 
purposes,  in  approximately  48  percent  of  the  public  school 
districts  (Becker,  1986;  Niman,  1987).  Many  states  now  require 
students  to  take  computer  literacy  classes.  Because  of  the  great 
influx  of  computers  into  schools,  the  time  has  come  to  assess  the 
impact  of  the  electronic  screen  on  reading  performance  in  a  school 
setting. 

Reading  Performance 

There  is  considerable  evidence  that  electronic  screens 
negatively  impact  reading  performance  when  presenting  text.  Adults 
read  more  slowly  from  electronic  screens  (Gould,  Alfaro,  Barnes, 
Finn,  Grischkowsky ,  &  Minuto,  1987;  Gould,  Alfaro,  Finn,  Haupt,  & 
Minuto,  li^87;  Gould  &  Grischkowsky,  1984,  1986;  Hansen,  Doring,  & 
WhDtlock,  1978;  Kathaway,  1984;  Kruk  &  Muter,  1984;  Muter, 
Latremouille,  Treurniet,  &  Beam,  1982).  A  few  studies  (Gould  & 
Grischkousky,  1984,  1986;  Muter  et  al.,  1982)  have  indicated  that 
reading  rate  decreased  by  as  much  as  20  to  30  percent  when  the 
computer-based  electronic  screen  was  the  mode  of  text  presentation. 

Text  Legibility 

Text  legibility  research  defines  those  features  that  affect 
electronic  screen  text  legibility.  According  to  Tinker  (1963) ,  a 
foremost  scholar  in  printed  text  legibility,  reading  speed  is 
critically  dependent  upon  text  legibility.  Legibility,  according 
to  Tinker  (1963,  p.  8), 

...deals  with  the  coordination  of  those  typographical 
factors  inherent  in  letters  and  other  symbols,  words,  and 
connected  textual  material  which  affect  ease  and  speed 
of  reading. 

The  factors  associated  with  legibility  include:  the  size  of  the 
print,  the  shape  of  the  font,  the  illuriination  of  the  reading 
environment,  the  color  and  quality  of  the  paper,  and  the  distance 
between  letters  and  between  lines  of  print.  Daniel  (1985,  p.  4) 
in  referring  to  electronic  screens  stated,  "the  term  *  legibility' 
denotes  how  specific  physical  characteristics  of  a  display  effect 
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[sic]  visual  fatigue,  reading  speed,  and  ultimately  comprehension" . 

Studies  of  printed  text  legibility  date  back  to  Sanford  (1888) 
and  Griffing  and  Franz  (1896) .  Huey  (1968)  summarized  many  text 
legibility  studies  and  made  recommendations  about  the  optimum  text 
presentation.  The  conclusions  from  Huey  (1968)  included:  (a) 
the  type  should  be  no  smaller  than  1.5  mm  or  6  points,  (b)  the 
optimal  line  length  should  be  between  90  and  100  mm  or  about  60 
characters  per  line,  (c)  the  best  color  paper  is  white,  and  (d)  the 
use  of  double  spaced  text  or  additional  leading,  that  is  more  space 
between  lines  of  print  improves  legibility.  Tinker  (1963)  also 
found  that  the  curvature  of  printed  material,  like  that  found  in 
^ooks  with  narrow  "gutters",  reduced  reading  speed. 

Finally,  Tinker  (1963)  investigated  color  combinations  of 
print  and  paper.  Tinker  (1963)  observed  that  black  print  on  a 
white  background,  "yielded  the  most  rapid  reading"  (p.  146).  But, 
other  color  combinations  were  almost  as  effective:  green  on  white, 
blue  on  white,  and  black  on  yellow.  All  other  color  combinations 
retarded  speed  of  reading  considerably. 

Computer  Screen  Legibility 

During  the  1970s  and  1980s,  researchers  investigated 
electronic  screen  legibility.  Hoover  (1977)  compared  reading 
performance  with  single  and  double  spaced  computer  text.  No 
statistically  significant  differences  were  found  in  reading  rate 
although  the  subjects  indicated  that  they  had  more  trouble  reading 
single  spaced  text  (Hoover,  1977). 

Kruk  and  Muter  (1984)  performed  a  series  of  three  experiments 
to  determine  if  it  was  feasible  to  read  continuous  text  from  an 
electronic  screen.  They  concluded  that  it  was  feasible,  but  that 
reading  speed  was  slowed  by  approximately  2  5  percent.  They  later 
concluded  that  this  speed  deficit  could  be  decreased  to  10.9 
percent  through  the  use  of  double  spacing. 

Other  studies  investigated  reading  rate  using  printed  text  and 
computer  screen  text.  Hansen  et  al.  (1978)  had  college  students 
take  a  reading  test  on  computer-based  electronic  screens  and  on 
paper.  They  found  that  students  read  more  slowly  on  computer 
screens.  Kak  (1981)  performed  a  similar  study  with  a 
standardized,  timed  reading  test.  Kak's  results  indicated  no 
statistically  significant  differences  in  reading  scores  when  only 
completed  parts  of  the  test  were  graded.  However,  a  four  percent 
slower  reading  speed  was  reported  for  students  taking  the  test  on 
computer.  Kak  (1981)  concluded  "that  legibility  criteria  may  be 
excessively  strict  when  applied  to  tasks  involving  reading  of 
connected  prose  text"  (p.  139). 
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Switchenko  (1984)  conducted  a  study  involving  adult  subjects 
in  a  comprehension  reading  task.  Subjects  read  two  short  articles 
which  differed  in  degree  of  difficulty.  Subject  then  answered 
comprehension  questions.  Switchenko  found  no  significant 
differences  in  reading  speed  or  comprehension  scores  from  computer 
or ^ paper.  Switchenko  (1984)  argued  that  studies  which  found  a 
difference  in  reading  speed  did  not  have  adequate  and  proper 
experimental  controls.  Gould  et  al.  (1987)  concluded  that  adults 
could  read  as  fast  from  a  computer  screen  as  they  could  read  from 
paper.  In  the  experiment,  the  fonts  appeared  similar  in  the  two 
jaedia.  An  extremely  high  resolution  computer  displays  with  black 
font  on  a  white  background  was  used.  Thus,  there  is  some 
controversy  whether  or  not  reading  speed  is  decreased  on  a  computer 
screen.  An  important  note  that  needs  to  be  made  is  that  all  of 
these  studies  used  adults.  To  this  point,  there  are  only  two 
studies  (Feldmann  &  Fish,  1987a,  1987b)  involving  school  aged 
children  assessing  reading  rate  and  comprehension  differences  with 
paper  and  computer  screens. 

Statement  of  the  Problem 

A  survey  of  available  literature  on  electronic  screen 
ility  uncovered  only  one  related  study  involving  junior  high 
e  students  (Feldmann  &  Fish,  1987b).  The  students  in  the  study 
s  <  high  ability  eighth  grade  students  and  no  statistically 
s.^^nificant  difference  in  reading  comprehension  was  found  between 
paper  and  computer-based  electronic  screen  media.  Other  electronic 
screen  legibility  studies  were  generally  conducted  in  the  workplace 
with  small  samples  of  convenient  adult  subjects. 

Other  questions  also  need  to  be  addressed.  If  indeed,  junior 
high  students  do  read  more  slowly  from  electronic  screens  as  adults 
seem  to,  are  there  easily  controlled  factors  which  might  improve 
the  legibility  of  print  on  electronic  screens? 

The  present  study  investigated  if  there  were  differences  in 
reading  rate  scores  of  eighth  grade  junior  high  students  on  a  cloze 
reading  comprehension  task  involving  the  reading  of  text  from 
several  different  modes  of  presentation.  The  following  modes  of 
presentation  were  studies: 

1.  An  electronic  screen  with  white  text  on  black  background 
utilizing  a  line  length  of  60  characters. 

2.  An  electronic  screen  with  white  text  on  a  black  background 
utilizing  a  line  length  of  iO  characters. 

3.  An  electronic  screen  with  black  text  on  white  background 
utilizing  a  line  length  of  60  characters. 

4.  An  electronic  screen  with  black  text  on  white  background 
utilizing  a  line  length  of  80  characters. 
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5.  An  electronic  screen  with  black  text  on  a  light  blue 
background  utilizing  a  line  length  of  60  characters, 

6.  An  electronic  screen  with  black  text  on  a  light  blue 
background  utilizing  a  line  length  of  80  characters, 

7 .  An  electronic  -creen  with  white  text  on  a  light  blue 
background  utilizing  a  line  length  of  60  characters, 

8.  An  electronic  screen  with  white  text  on  a  light  blue 
background  utilizing  a  line  length  of  80  characters. 

9.  White  paper  with  black  IBM  PC  Junior  matching  font  printed 
on  a  dot  matrix  printer  with  a  line  length  of  60  characters, 

10.  White  paper  with  black  Courier  10  font  printed  on  a  laser 
printer  with  a  line  length  of  60  characters. 

The  two  research  questions  addressed  in  this  study  were: 

1,  Is  there  a  difference  in  eighth  grade  students'  reading 
rate  on  a  reading  comprehension  task  among  groups  performing  the 
cask  using  several  different  modes  of  presentation? 

2,  Is  there  a  difference  in  eighth  grade  student's 
comprehension  score  on  the  reading  comprehension  task  amopg  groups 
performing  the  task  using  several  different  modes  of  presentation? 

Method 

Three  hundred  junior  high  school  students  in  computer  literacy 
classes,  in  seven  schools  from  a  large  suburban  school  system,  were 
involved  in  this  study.  The  subjects  were  randomly  placed  into  one 
of  10  treatment  groups.  The  subjects  completed  a  cloze  reading 
passage.  The  first  and  last  sentences  of  the  passage  were  left 
intact,  but  every  fifth  word  in  the  main  body  of  the  passage  was 
deleted.  The  passage  had  fifty  missing  words.  The  students  read 
the  passage,  decided  which  words  were  missing,  and  wrote  these 
words  on  an  answer  sheet.  The  time,  in  minutes  and  seconds,  !:aken 
to  complete  the  exercise  was  measured  by  the  investigator  using  a 
stop  watch.  The  time  to  complete  the  passage  was  used  as  the 
subject's  reading  rate.  The  passages  were  subsequently  scored,  and 
this  score  was  used  as  the  subject's  comprehension  measure. 
Subjects  all  read  the  same  passage,  but  the  passage  was  presented 
-Lii  one  of  the  10  different  text  presentation  modes.  This  design 
allowed  comparisons  to  be  made  among  the  means  of  the  various 
treatment  groups. 

In  order  to  allow  the  subjects  to  become  familiar  with  the 
cloze  procedure,  subjects  participated  in  a  20  minute  practice 
session  one  day  before  the  data  collection.  The  practice  session 
also  enabled  the  subjects  to  practice  using  the  arrow  keys  on  the 
computer  to  scroll  text  forward  and  backward  on  the  screen.  Actual 
collection  of  data  took  one  class  period.  Administration  of  the 
cloze  passage  took  approximately  30  to  35  minutes.     Collection  of 
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materiels  took  approximately  5  minutes  with  no  more  than  45  to  50 
minutes  was  needed  to  collect  the  data.  Class  periods  in  the 
schools  participating  in  the  study  were  55  minutes  in  length. 

The  students  were  told  that  their  reading  would  be  timed,  but 
that  reading  time  was  not  as  important  as  being  accurate.  Students 
v/ere  asked  to  rnise  their  hand  when  they  completed  the  exercise. 
A  stop  wav  ::h  was  used  to  measure  the  time  it  took  students  to 
complete  the  cloze  passage.  When  all  students  had  completed  the 
exercise,  all  materials  were  collected. 

Instrumentation 

The  De  Santi  Cloze  Reading  Inventory  was  used  as  the  dependent 
variable.  The  cloze  procedure  may  be  used  to  determine  if  students 
are  able  to  comprehend  certain  material*  Three  levels  are  used  in 
traditional  measures  of  comprehension  for  cloze  passages: 
independent,  instructional,  and  frustration.  Fifty  responses  are 
possible  in  the  passage  and  the  number  of  correct  responses  is 
multiplied  by  .02  to  derive  a  percentage. 

"The  inventory  was  field  tested  3,456  times  with  a  total  of 
864  students  drawn  from  two  school  systems  (De  Santi,  Casbergue, 
&  Sullivan,  1986,  p.  131).  The  correlated  concurrent  and 
predictive  validity  for  eighth  grade  cloze  passages  with  CTBS  total 
reading  scores  was  found  to  be  statistically  significant  at  the 
.001  level. 

The  inventory  has  single-rater  and  inter-rater  reliability. 
De  Santi  and  Sullivan  (1985)  determined  single-rater  reliability; 
the  mean  reliability  of  the  Traditional  Comprehension 
interpretation  value  was  determined  to  be  .96.  This  was  also  the 
Kaan  value  for  the  inter-rater  reliability  (De  Santi  &  Sullivan, 
1985)  . 

For  the  purposes  of  this  study.  The  De  Santi  Cloze  Reading 
Inventory  was  converted  to  the  eight  different  computer 
presentation      modes.  Additionally,      two      different  paper 

presentations  modes  were  created.  The  first  paper  version  (group 
9)  was  created  using  the  ChiWviter  (1988)  software  package  to 
create  a  font  simila^  to  the  font  used  on  the  IBM  PC  Junior 
computer  screen.  The  second  paper  version  (group  10)  was  printed 
on  a  laser  printer  with  Courier  10  font.  These  modifications  were 
made  to  pldce  better  controls  on  the  presentation  modes. 
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R6 suits 

The  results  of  the  two  research  questions  are  presented • 

1.  Is  there  a  difference  in  eighth  grade  students'  reading 
rate  on  a  reading  comprehension  task  among  groups  performing  the 
task  using  several  different  modes  of  presentation? 

To  test  this  first  hypotheses,  a  balanced  one  way  Analysis  of 
Covariance  (iVNCOVA)  statistical  procedure  was  performed  using 
SPS3PC.  The  reading  rate  of  subjects  on  the  cloze  reading  task  was 
the  dependent  variable  while  the  covariate  was  the  normal  curve 
equivalence  (NCE)  score  on  the  Total  Reading  subtest  of  the  CTBS. 
Table  1  presents  a  summar^^  of  descriptive  statistics  for  the  CTBS 
covariate  scores.  The  highest  score  for  the  sample  test  was  99  and 
the  lowest  score  was  3. 

INSERT  TABLE  1  ABOUT  HERE 

Subjects  were  randomly  assigned  by  the  use  of  a  random  number 
table  to  10  text  presentation  groups.  The  mean  reading  rate  for 
each  group  was  determined.  Table  2  presents  the  mean  and  standard 
deviation  for  the  reading  rate  of  the  10  groups. 

INSER":^  TABLE  2  ABOUT  HERE 

No  statistically  significant  differences  were  found  at  the  .05 
level  in  the  omnibus  hypothesis  among  the  means.  Additionally,  the 
effect  size  was  low  (4.5%),  thus  the  results  were  also  not 
educationally  significant.  Table  3  presents  the  ANCOVA  summary 
table  for  reading  rate.  The  null  hypothesis  was  not  rej(*xted; 
therefore,  no  post  hoc  tests  were  performed.  Thi?  finding 
contradicts  most  research  studies  (Granaas,  McKay,  Laham,  Hurt,  & 
Juola,  1984;  Heppner,  Anderson,  Farstrup,  &  Weiderman,  1985). 

INSERT  TABLE  3  ABOUT  HERE 

When  performing  a  cloze  reading  exercise,  the  reading  rate  of 
eighth  grade  students  was  not  statistically  or  educationally 
significant  when  reading  from  paper  or  computer.  Although  no 
statistical  differences  were  found  with  the  ANCOVAs,  students  with 
independent  level  reading  ability  did  read  27  percent  slower  from 
electronic  screens  than  paper. 
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comprehension  score  on  the  reading  comprehension  task  among  groups 
performing  the  task  using  several  different  modes  of  presr  itation? 

To  test  the  ^>econd  null  hypothesis,  a  balanced  one  way  ANCOVA 
statistical  procedure  was  performed  using  the  comprehension  of 
subjects  on  the  cloze  reading  task  as  the  dependent  variable.  The 
covariate  was  the  normal  curve  equivalence  score  (NCE)  on  the  Total 
Reading  subtest  of  the  CTBS .  A  mean  comprehension  score  was 
determined  for  oach  group.  Table  2  presents  the  descriptive 
statistics  for  the  comprehension  scores. 

No  statistically  significant  differences  were  found  among  the 
means  on  the  omnibus  hypothesis.  Table  4  presents  the  ANCOVA 
summary  table  for  comprehension  scores.  The  null  hypothesis  was 
not  rejected  and  thus,  no  post  hoc  tests  were  performed. 

Students'  comprehension  scores  did  not  appear  to  differ  for 
any  of  the  groups  tested.  This  finding  was  in  agreement  with  the 
vast  majority  of  previous  studies  which  tested  comprehension  in 
various  text  modes  (Feldmann  &  Fish,  1987a,  1987b,  1988;  Gould  et 
al.,  1987;  Gould  &  Grischkowsky ,  1984,  1986;  Kruk  &  Muter,  1984; 
Kak,  1981;  Switchenko,  1984). 

INSERT  TABLE  4  ABOUT  HERE 

Conclusions 

The  findings  of  no  differences  in  reading  speed  from 
electronic  screens  to  paper  conflicts  with  most  of  the  findings  '  n 
adult  reading  rate  research  (CoulJ  et  al.,  1987;  Gould  et  al., 
1987;  Gould  &  Grischkowsky,  1984,  1986;  Hansen  et  al . ,  1978; 
Hathaway,  1984;  Kruk  &  Muter,  1934).  Reasons  accounting  for  the 
difference  in  findings  may  be:  (a)  the  reading  of  the  Luult 
subjects  was  superior  to  the  eighth  grade  students;  (b)  the  nature 
of  the  reading  materials,  proofreading  as  opposed  to  a  cloze 
passage;  and  (c)  the  focus  on  groupo  of  students  rather  than  the 
individualized  treatment  of  the  adults.  This  would  suggest  that 
reading  level  might  be  a  factor  affecting  reading  rate  froia 
electronic  screens  or  paper. 

The  finding  of  no  difference  in  comprehension  was  consistent 
with  previous  studies.  Thus,  the  ability  to  comprehend  text  is  not 
necessarily  a  lunction  of  legibility  but  deciding  level.  The 
reading  rate  is  affected  by  legibility.  Thur,  a  reader  would 
reduce  his  reading  rate  in  order  to  maintain  comprehension. 
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Summary 

There  was  no  statistically  significant  difference  in  eighth 
grade  students  reading  from  electronic  screen  or  paper.  The  length 
of  lines  or  the  medium  seemed  to  make  no  difference.  Thus,  for  a 
30 ^minute  reading  task,  the  medium  did  not  make  a  marked  difference 
in'either  the  reading  rate  or  comprehension. 

More  research  needs  to  be  performed  with  school  aged  children 
to  bett5*.r  identify  if  different  ability  groups  of  students  are  more 
or  less  affected  by  the  presentation  mode  of  text.  Research  should 
also  be  performed  to  investigate  if  the  reading  rate  and/or 
comprehension  would  deteriorate  given  longer  reading  spans. 
Finally,  other  computers  besides  the  IBM  PC  Junior  should  be  used 
to  determine  if  particular  computers,  and  the  generated  font, 
affects  reading  rate  and/ or  comprehension. 
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Table  1 

Means  and  Standard  Deviations  for  CTBS 
Total  Reading  NCEs  by  Group 


Group 

Mean 

SD 

1 

45.62 

19.70 

2 

56.93 

21.46 

3 

55.03 

17.73 

4 

52.40 

17  .39 

5 

55.13 

23  .88 

6 

55.77 

19.10 

7 

57.50 

22  .33 

8 

52.23 

21.99 

9 

49.70 

18.68 

10 

51.47 

23  .10 

Total 

54.08 

20.48 

Note.  Each 

group  contained  an 

Table 

Descriptive  Statistics  by  Group  on 
Reading  Rate  and  Comprehension 


Group 

Readina  Rate 

SD 

Comprehension 

SD 

Mean 

Mean 

1 

12.28 

3  .84 

49.80 

9.  00 

2 

14.10 

5.57 

50.40 

8.43 

3 

13.36 

4  .34 

50.87 

8.  55 

4 

13.13 

3.30 

45.53 

7.25 

5 

12.90 

3  .66 

50.73 

7.17 

6 

12.16 

4.03 

48.53 

9.67 

7 

13. 8o 

5.30 

48.47 

9.77 

8 

13.02 

3.68 

46.80 

12.29 

9 

11.96 

4  .05 

50.13 

8.40 

10 

12.78 

6.05 

48.13 

■'0.53 

Total 

12.96 

4  .45 

48.94 

9.28 

Note.  Reading  rate  was  measured  in  minutes. 
Note.  Each  group  contained  an  N  of  30. 
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Table  3 

ANCOVA  summary  Table  for  tho.  Effect  of  Text  Mode 
Presentation  on  Reading  Rate 


Source 


ss 


df 


MS 


Covariate 
NCE 

Main  Effect 
Group 

Residual 


1154.78 

205.31 
4574.2/ 


9 

289 


1154.78  72.96 


22.81 
15.83 


44 


.195 


.035 


Table  4 

ANCOVA  Summary  Table  for  the  Effect  of  Text  Mode 
Presentation  on  Comprehension 


Source 


SS 


df 


MS 


Covariate 
NCE 

Main  Effect 
Group 

Residual 


10441. 34 

78. 10 
145L1. 49 


9 

289 


10441.34  207.37 


8.68 
50.35 


.  17 


.416 


003 
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Feminist  scholiTsbip  snd  feminist  activism  proceed  not  through  a 
st&rile.pIsnMT  dialectic  of  thesis,  aaaly^.  synthesis,  but  through  a 
dynamic  process  of  unthinking,  rett  inking,  energizing,  and 
transforming.  At  its  best,  feminism  creates  new  iife  forms  out  of 
experiences  as  common  as  seawater  and  insights  as  electrifying  as 
lightening.  (Bush.  1983.  p.  152) 

The  purpose  of  this  paper  is  to  begin  a  feminist  unthinking- 
rethinking-energizing-transforming  of  the  project  (phenomenon,  activitiy, 
effects,  etc.)  we  call  "educational  technology. "  As  Corlann  Gee  (Corky)  Bush 
indicates  in  the  statement  above,  the  activity  of  unthlnkmg  requires  that  we 
resist  the  traditional,  habitual,  and  for  many  of  us  almost  automatic, 
tendency  to  begin  an  eiamination  of  "educational  technology"  (the  thesis)  by 
breaking  it  into  constituent  parts,  "education"  and  "technology",  analyzing 
these  separately,  and  rationalizing  their  synthesis.  An  unthinking  of 
Educational  Technology  requires  that  we  unthink  this  procedure  which  is 
inherent,  not  only  in  our  instructional  design  models,  but  also  in  many  of  our 
critiques  (e.g..  Nunan.  1983),  and  in  those  studies  /hich  attempt  to  relate 
specific  technologies  to  educational  goals.  Every  analysis  based  upon  the 
independent  listing  and  correlation  of  objectives  with  media  and  methods 
has  the  effects  of  reifying  some  set  of  educational  goals  or  objectives  and  of 
validating  some  set  of  media,  materials,  and  hardware  or  software 
technologies  as  the  lexical  completion  of  the  reified  goals. 

Such  an  analysis  fragments  the  domain  of  discourse,  changing  its  focus 
from  educational  technology  perse  to  a  set  of  concerns  about  the  elements 
identified  in  the  analysis.  Many  of  these  are  relevant  to  the  interests  of 
women;  for  example,  the  question  of  whether  computers  are  less  appealing 
and  effeaive  for  females  than  lor  males  is  of  some  imr  ^tance  to  women 
seeking  equal  opportunities  to  learn  specific  content,  but  it  begs  the  question 
of  what  we  mean  by  "effective."  Moreover,  each  analysis  invites  further 
analysis  and  further  fragmentation;  having  identified  a  specific  medium  or 
category  of  materials,  we  can  ask  more  specific  questions:  how  is  sexism 
replicated  in  that  medium?  how  might  the  sexist  elements  identified  be 
eliminated? 

The  point  is  that  in  this  type  of  examination  we  are  deeply  engaged  in 
ihinking  edocttiona!  teclmolGgy;  at  each  step  of  the  analytic  procedure, 
the  previous,  more  inclusive,  construct  is  accepted  and  implicitly  valorized. 
The  higher  order  construct  shapes  the  ways  in  which  we  think  about  its 
parts  and  the  questions  which  we  form.  Therefore,  if  we  wish  to  unthi&k 
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educational  technology  (and  I  do),  we  must  resist  the  tendency  to  break  it 
into  parts.  This  "first  unthinking"  reveals  that  not  only  does  educational 
technology  (through  ISD  models)  force  upon  us  a  fragmented  view  of  the 
educational  process  and  of  the  content  or  topics  of  education,  but  also  the 
field  tends  to  split  itself  apart  for  purposes  both  of  research  and  of  self- 
analysis.  If  we  are  to  unthink  educational  technology  we  must  struggle  to 
deal  with  it  as  a  coherent  whole. 
A  holistic  ¥iew  of  educationil  tectmolog^T 

The  problem  of  describing  what  educational  technology  is  has  been 
addressed  by  AECT.  resulting  in  a  lengthy  definition: 

"Educational  Technology"  is  a  complex,  integrated  process  involving  people, 
procedures,  ideas,  devices,  and  organization  for  analyzing  problems,  and 
devising,  implementing,  evaluating,  and  managing  solutions  to  these  problems, 
involved  in  all  aspects  of  human  learning.  In  educational  technology,  the 
solutions  to  problems  t«ke  the  form  of  all  "Learning  Resources"  that  are 
designed  and/or  selected  as  Messages,  People,  Materials,  Devices,  Techniques, 
and  Settings.  The  processes  for  analyzing  problems  and  devising, 
implementing,  ind  evaluating  solutions  are  identified  by  the  "Education- 
Development  Functions"  of  Kesearch-Theory.  Design,  Production,  Evaluation, 
Selection,  Logistics,  and  Utilization.  The  processes  of  directing  or  coordinating 
one  or  more  of  these  functions  are  identified  by  the  "Educational  Management 
Functions"  of  Organization  Management  and  Personnel  Management. 
Educ»Uon»l  Technology:  Definition  and  Glossuy  of  Terms,  1977 

Implicit  in  this  definition,  although  it  is  not  explicitly  recognized,  is  the  idea 
of  systems: 

. . .  systems  can  be  defined  as  deliberately  designed  synthetic  organisms, 
comprised  of  intemlated  and  interacting  components  Thich  are  employed  to 
function  in  an  integrated  fashion  to  attain  some  predetermined  purpose. 
Therefore,  the  best  vay  to  identify  a  system  is  to  reveal  its  purpose.  (Banathy, 
i%8,  pp.  2-3) 

Taken  together,  these  definitions  confirm  the  essential  character  of 
educational  technology  as  having  to  do.  not  with  existing  holistic  phenomena, 
but  rather  with  a  multitude  of  ideas,  agencies,  procedures,  and  artifacts 
which  are  brought  together  and  integrated  to  a*eate  "solutions  to  problems." 
Although  tht;  source  and  nature  of  the  problems  is  unspecified,  the  means  of 
dealing  with  them  and  of  evaluating  their  solutions  are  listed  (somewhat 
cryptically,  using  terms  which  are  separately  defined  in  the  AECT  glossary). 
The  problems  to  be  solved  through  educational  technology  must,  then,  be 
those  problems  which  lend  themselves  to  potential  solution  and  solution 
assessment  through  integration  of  the  listed  mechanisms  Into  "deliberately 
designed  synthetic  organisms."  The  AECT  definition  specifies  no  particular 
purpose  for  the  systems  created;  the  purpose  must  be  assumed  to  arise 
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from  the  context  in  which  educational  technology  exists,  that  is,  from 
patriarchy.  By  its  own  definition,  then  Educational  Technology  is 
reminiscent  of  the  new  hammer  in  the  hands  of  the  young  kid  who  suddenly 
sees  everything  as  "needing  hammering. " 

While  the  definition  of  educational  technology  8|>ecifie8  little  about 
purpose,  it  provides  a  wealth  of  information  about  elements  to  be  used  in 
the  design  and  development  of  applications  of  the  tools  and  techniques  of 
the  technology.  Beginning  with  the  statement  that  Educational  Technology  is 
a  process,  the  definition  specifies  the  elements  of  the  process,  the  forms  of 
the  problem  solutions,  and  the  assignment  of  tasks  to  the  constituent  parts 
or  practitioners  of  the  field.  Analogously  to  the  ways  in  which  science  is 
defined  by  "the  scientific  method(s)."  educational  technology  is  defined  by 
its  processCes).  And,  analogous  to  the  claim  of  science  to  seek  an 
understanding  of  our  universe  in  ail  its  aspects  is  the  definitional  claim  of 
educational  technology  to  deal  with  problems  "involved  in  all  aspects  of 
human  learning."  Continuing  with  this  analogy,  feminist  unthinking 
uncovers  ways  in  which  the  femuiist  critique  of  science  can  inform  a 
rethinking  of  educational  technology. 

Speaking  from  diverse  feminist  perspectives,  a  number  of  feminists 
have  addressed  the  questions  of  whether  there  can  be  a  feminist  science 
and.  if  so,  how  it  would  be  charaaerized.  Among  these  thinkers/unthinkers, 
epistemologist  Sandra  Harding  (1986,  1987)  has  not  only  critiqued 
malestream  science  from  several  perspectives,  but  has  also  identified  three 
essential  characteristics  of  a  feminist  approach  to  science:  ( 1 )  the  theorizing 
of  gender  as  a  variabli?  of  consequence,  (2)  the  valuing  of  women  s 
experience  as  a  scientific  resource,  and  (3)  the  positioning  of  the  researcher 
m  the  same  critical  plane  as  the  researched.  Analogues  to  Harding's 
observations  make  visible  three  areas  for  feminist  rethinking  of  educational 
technology.  Attention  of  the  field  is  directed  by  its  definition  to  "all  aspects 
of  humait  learning '  rather  than  toward  learning  by  real  live  people  who 
differ  in  ways  (including  gender)  which  are  consequential.  Moreover, 
people  are  among  the  Learning  Resources  that  can  be  "designed  and/or 
selected;"  the  specifically  human,  specifically  gendered,  and  specifically 
personal  is  ignored  if  not  suppressed  in  educational  technology.  Secondly, 
educational  technology  clearly  ignores  or  rejects  the  experiences  of  women 
as  the  majority  of  teachers  and  as  the  fifty-or-so  percent  of  learners  who 
have  experienced  "human  learning"  in  educational  settings.  (I  shall  return  to 
these  points  later.)  Thirdly,  educational  technology  is  clearly  a  "top-down" 
activity;  the  definition  of  the  field  indicates  that  the  Learning  Resources, 
Education-  Development  Functions,  and  Educational  Management  Functions 
which  comprise  the  process  of  educational  technology  exist  independently 
and  apart  from  any  specific  learner  or  group  of  learners.  In  sum,  the 
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feminist  e piste mologicai  critique  of  science  suggests  several  important 
avenues  for  the  unthinking  of  educational  technology. 

The  ecofeminist  critique  of  malertream  science  brings  feminist 
unthinking  to  science  and  technology,  not  from  the  perspective  of 
epistemology.  but  from  that  of  women  concerned  about  the  environment. 
Carolyn  Merchant  (1980),  one  of  the  "mothers"  of  ecofeminism  frames  her 
historical  analysis  of  science  and  technology  in  the  set  of  concerns  common 
to  environmentalists  and  feminists,  stating  "we  must  reexamine  lue 
formation  of  a  world  view  and  a  science  that,  by  reconceptualizing  reality  as 
a  machine  rather  than  a  living  organism,  sanaioned  the  domination  of  both 
nature  and  woman  (p.  ivii)."   Not  only  have  ecofeminists  continued  to  be 
engaged  in  the  unthinking  which  Merchant  began,  but  they  are  also  engaged 
in  rethinking  and  transforming  science  and  technology.  As  Irene  Diamond 
and  Lee  Quinby  (1988)  put  it,  "In  place  of  current  scientific  theories  and 
practices  imbued  with  questionable  notions  of  certainty,  objectivity,  and 
domination,  ecofeminist  discourse  emphasizes  indeterminacy, 
interconnectedness,  and  nurturance  (p.  203)  "  Ecofeminist  considerations 
invite  us  to  consider  whether  educational  technology  perceives  the  reality  of 
"all  aspects  of  human  learning"  as  more  like  a  free-standing  machine  than  a 
living  social  organism,  and  to  unthink  this  perception.  How  are  educational 
technology  practices  of  "analyzing  problems  and  devising,  implementing, 
evaluating,  and  managing  solutions"  rooted  in  more  general  notions  of 
certainty,  objectivity,  and  domination?  How  do  these  practices  sanction  the 
domination  of  both  nature  and  women  (and  men)?  Can  we  rethink 
educational  technology  in  ways  that  emphasize  indeterminacy  and  the 
uncertainty  of  all  "human  learning,"  interconnectedness  of  (school)  learning 
and  lived  experience,  and  nurturance  of  "the  learner"  as  a  real  live  gendered 
individual  person? 

Thus,  feminist  epistemologies  and  ecofeminism  (together  with  other 
feminist  theorizing)  provide  vocabularies,  analogies,  insights,  and  energy  for 
the  unthinking  of  educational  technology  as  a  process  for  the  design  and 
development  of  educational  interventions,  learner  deficit  models,  negative 
reinforcement  strategies,  and  other  "Learning  Resources"  many  of  whose 
very  names  invite  us  to  continue  feminist  unthinking  of  the  definition  and 
components  of  educational  technology.  Despite  the  importance  of  these 
issues,  they  do  not  encompass  feminist  concerns  with  regard  to  educational 
technology,  few  these  issues  all  reside  within  the  narrow  view  of  educational 
technology  as  an  isolated  endeavor  which  might  be  improved,  if  not 
perfect?d,  in  and  of  itself.  More  important  issues  arise  from  the  examination 
of  educational  technology  in  other  contexts  and  from  other  perspectives.  In 
the  remainder  of  this  paper  I  shall  turn  to  one  of  these  contexts,  that  of  the 
user. 
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Educational  Tectmology  and  its  Users. 

Feminist  critiques  of  teclinologies  have  contributed  substantially  to 
tlie  technological  literature  through  their  examination  of  the  effects  of 
innovations  on  the  practices  and  the  lives  of  technology  users,  particularly 
when  those  users  are  women.  It  is  instructive  to  the  unthinlcing  of 
educational  technology  to  consider  an  example  of  this  work  which  has  been 
especially  effective  in  unthinking  the  media  "hype '  which  promotes 
household  technologies  with  promises  of  "less  work  for  mother."  Feminist 
unthinking  of  the  notion  that  the  automatic  washing  machine  is  purely  and 
simply  a  boon  to  women  has  uncovered  numerous  effects  on  the  lives  of 
women  in  addition  to  the  obvious  fact  that  washing  any  given  load  of  clothes 
is  a  lot  easier  than  it  was  (Cowan.  1 983).  As  a  direa  result  of  the  new 
technology,  the  activity  of  washing  clothes  is  no  longer  a  scheduled  activity 
restricted  to  a  particular  day.  but  takes  place  "as  needed."  Generally 
speaking,  people  have  more  clothes  in  need  of  washing  and  in  need  of  special 
attention  as  they  are  laundered.  Standards  for  laundering  have  changed, 
requiring  "colors  brighter  and  whites  whiter  than  white  '  Gothes  washing 
today  ib  a  solitary  activity,  rather  than  the  peer  group  or  mother-children 
activity  it  was  in  the  past.  Thus,  the  automatic  washing  machme  has 
changed  substantialy  the  daily  lives  of  women,  imposing  new  standards, 
schedules,  and  structures  on  them;  at  the  same  time,  the  "social  credit"  for 
laundering  has  been  denied  women  by  the  notion  that  the  automatic  washer, 
not  the  woman  who  operates  it,  takes  care  oS  the  wash. 

Educational  Technology  invites  and  requires  unthinking  from  the  point 
of  view  of  the  user  analogous  to  the  unthinking  that  feminists  have  applied 
to  household  and  office  terminologies.  Such  unthinking  might  begin  with  the 
examination  of  both  the  purported  benefits  of  the  technology  for  the  user 
and  the  changes  which  the  technology  imposes  on  the  users  with  respect  to 
standards,  schedules,  social  structures,  and  social  credit .  As  we  begin  this 
examination,  however,  a  prior  question  arises:  Who  are  the  primary  users  of 
educational  technology?  Several  categories  of  users  are  apparent:  school 
systems  and  their  administrators,  teachers,  children  in  classrooms, 
independent  students,  military  trainers,  job  applicants,  trainees  in  business 
and  industry,  and  a  host  of  others  who  are  engaged,  in  one  way  or  another, 
in  seeing  to  it  that  some  (particular)  "human  learning"  takes  place.  While 
recognizing  that  this  diversity  of  users  does  affect  the  nature  of  the 
technology,  I  shall  consider  only  the  effects  of  educational  technology  on 
users  in  schools. 

Within  the  context  of  schools  t!:ere  are  three  clearly  separated 
categories  of  major  users  of  educational  technology:  school  administrators, 
teachers,  and  students.  Administrators  (superintendents,  curriculum 
supervisors,  prmcipals)  use  educational  technology  as  means  of 
implementing  local,  state,  and  national  educational  policies,  of  standardizing 
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instruction  and  evaluation,  of  realizing  certain  measurable  efficiencies,  and 
of  meeting  parental  and  community  demands  that  schools  be  "modern"  and 
"effective."  Teachers  use  educational  technology  partly  as  a  result  of 
administrative  mandates  and  partly  through  the  selection  of  instructional 
media  and  materials  for  particular  goals  and  topics  of  the  curriculum. 
Students  are  the  "end  users"  of  educational  technology  and  use  it  most  often 
as  directed  by  teachers.  Thus,  school  administrators  (primarily  male)  impose 
educational  technology  on  teachers  (primarii/  female):  in  turn,  teachers 
impose  educational  technology  on  students  of  both  seies.  Although  there  is 
a  need  to  unthink  the  idea  that  administrators  are  "free  agents"  in  this 
hierarchy  of  use.  I  will  focus  on  the  teacher  and  stu  lent  as  users  of 
educational  technoiog--' 

Teachers  as  users    educational  technology.  A  femiiiist  unthinking  of  the 
effects  of  educational  technology  on  teachers  must  begin  with  the 
observation  that  currently  and  historically  the  overwhelming  majority  of 
U.S.  public  school  teachers  of  kindergarten  through  12th  grade  are  females. 
Indeed,  school  teachmg  has  been  closely  identified  with  women  since 
Catherine  Beecher's  1846  identification  of  teaching  as  'woman's  true 
profession;"  until  the  current  wave  of  ftJiinism,  the  occupational  choices 
available  to  women  were  three:  secretary,  nurse,  and  teacher.  In  this 
statistical  and  historical  context,  current  criticism  of  school  teachers,  whether 
it  come  from  the  Holmes  Group's  (1986)  arguments  for  reform  of  teacher 
education  or  from  Heinich's  (1988)  arguments  for  replacing  teachers  with 
educational  technology,  must  be  viewed  at  least  in  part,  as  suppression  of 
whatever  unique  qualities  and  values  women  currently  bring  to  the  activity 
of  teaching.  Exactly  what  qualities  these  are  is  not  fully  known,  although 
nurturance  and  "the  continuation  of  mothering"  are  among  them  (Grumet, 
1988;  O'Brien.  1989;  Laird.  1988).  Further  feminist  study  is  needed,  for 
amidst  the  growing  hterature  on  feminist  pedagogy  at  the  post -secondary 
level  (e.g.,  Bunch  and  Pollack.  1983;  CuUey  and  Portuges.  1985).  there  is  a 
curious  and  serious  shortage  of  feminist  studies  of  teaching  of  the  K-12 
grades.  Major  exceptions  are  the  books  of  Grumet  (1988)  and  Weiler  (1988), 
both  of  which  highlight  the  importance  of  the  individual  teacher  listening  to 
the  voices  of  individual  students  as  she  practices  the  art,  not  only  the 
science,  of  teaching. 

Like  many  other  technologies.,  educational  technology  has  been 
developed  largely  outside  the  domain  of  the  user,  in  this  case  the  teacher, 
and  without  the  benefit  of  her  wisdom  as  to  how  technological  advance 
might  help  in  the  pursuit  erf  her  art.  Instead,  the  products  and  processes  of 
educational  technology  are  delivered  in  fully  developed  form,  sometimes 
with  mandates  for  their  use  and  sometimes  simply  as  materials  which  might 
be  useful.  Although  teachers  and  teacher  magazixies  complain  of  it,  I  am 
unaware  of  studies  of  how  this  lack  of  communication  effects  the  classroom 
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usability  of  these  products.  However,  feminist  studies  of  comparable 
practices  in  relation  to  office  computerization  clearly  indicate  that,  had  the 
women  workers  been  consulted  during  the  design  of  the  specific  systems 
they  are  required  to  use,  the  result  would  have  been  systems  which  were 
more  appropriate  both  for  the  iir  mediate  user  and  for  achieving  the  goals  of 
the  implementation  (Suchman  and  Jordan.  1988;  Zuboff,  1988).  An 
unthinker  needs  to  ask  why  teachers  are  not  major  actors  in  the 
specification,  design,  and  development  of  educational  technology.  Is  their 
exclusion  simply  an  example  of  what  Mary  O'Brien  (1989)  sees  as  the 
continued  exploitation  of  teachers  by  the  establishment  because  they  are 
women?  Is  it  an  extension  of  Barbara  Carson's  (1988)  observation  that  "The 
underlying  premise  of  modern  automation  is  a  profound  distrust  of  thinking 
human  beings "  (p.26 1 )?  Or  is  it  that  the  goals  of  educational  technology  are 
so  disc*ete  from  the  goals  of  teachers  that  the  experiences  of  teachers  are 
irrelevant  to  the  design  of  educational  technology? 

If  the  design  process  of  educational  technology  denies  the  value  of  the 
experiences  of  the  teacher  as  an  information  resource,  the  implementation 
processes  effect  the  structures,  standards,  and  schedules  by  which  she  plans 
and  performs  her  work.  Computer  managed  instruction  and  competency 
based  testing  elevate  the  importance  of  some  types  of  learning  while  limiting 
the  teachers'  choices  and  involvement  within  the  instructional  process;  thus, 
they  deny  the  value  of  any  wisdom  cm-  insight  she  may  have  gained  from  her 
years  of  teaching.  The  possibilities  for  flexible  planning  of  activities  which 
are  finished  only  when  students  reach  some  sort  of  closure  on  the  topic  are 
diminished  by  the  need  to  use  computer  labs,  broadcast  television,  or  scarce 
hardware  on  an  often  fragmented  schedule  not  of  her  making;  chaos  and  the 
feeling  that  nothing  is  ever  completed  is  introduced  into  the  teachers'  life. 
The  inability  to  preview  materials  as  students  will  experience  them,  either 
because  of  their  prior  unavailability  or  because  complex  branching  precludes 
any  examiniation  of  the  full  range    possibilities,  renders  the  teacher  less 
knowledgable  with  respect  to  students'  actual  experiences  and  at  a  loss  for 
helping  students  who  experience  difficulty  making  connections  or  who 
encounter  any  sort  of  software  failure. 

In  short,  the  structures,  standards,  and  schedule  of  the  teacher's 
school  day  become  out  of  her  control.  The  delivery  of  information,  and  the 
modelling  of  the  use  of  this  information,  are  increasingly  removed  from  her; 
often  the  information  provided  to  her  students  is  not  even  readily  available 
to  her.  Research  on  the  effectiveness  of  educational  technology  denies  the 
importance  of  her  work. " me ta- analyzing  "  her  contributions  out  of  the 
picture.  As  many  writers  on  the  future  of  schooling  have  observed,  the 
teacher  is  increasingly  a  manager  and  facilitator.  Her  position  becomes  very 
much  like  that  of  the  woman-as-laundress  of  our  example.  As  schedules  of 
instruction  become  more  complex,  as  the  standards  for  student  acquisition  of 
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"competencies"  increase  in  number,  and  as  expert  systems.  ICAI.  interactive 
video,  and  other  new  technologies  decrease  the  teacher's  familiarity  with  the 
topics  of  instruction,  apparently  she  will  still  be  charged  with  the 
management  of  'human  learning  activities;"  that  is,  she  will  be  responsible 
for  seeing  that  "human  learning "  takes  place.  Will  she  receive  sociai  credit 
for  teaching  when  students  learn?  Or,  will  "educational  technology"  take 
care  of  that? 

Students  as  users  of  educational  technologv.  In  thinking/unthinking  about 
students  as  users  of  educational  technology,  I  shall  focus  my  attention  on 
female  students.  However,  a  few  general  comments  are  important.  The 
language  of  educational  technology  tends  to  deny  the  essence  of  the  real  live 
person  who  is  the  "end-user"  of  the  technology.  No  longer  a  student  (who 
studies),  this  person  is  positioned  within  educational  technology  as  "the 
learner."  (Taylor,  1987).  The  generic  learner  does  not  behave  but  exhibits 
behaviors,  is  not  able  but  has  capabilities,  does  not  look  at  things  but  is 
presented  with  stimulus  material,  does  not  perform  but  meets  criteria,  anc 
so  on:  in  short,  "the  learner"  is  positioned  as  non-autonomous  and  passive  by 
the  language,  attitude  and  rigors  of  educational  technology.  Similar  positions 
are  held  by  army  recruits  enduring  the  rigors  of  boot  camp  and  by  fraternity 
pledges  undergoing  hazing;  it  is  a  male  position.  Sally  Hacker  (1989),  in  her 
feminist  examination  of  the  commonalities  of  military  training  and 
engineering  education,  theorizes  the  importance  of  such  positioning  to  the 
continuance  and  reproduction  of  patriarchy.  Further  unthinking  which 
extends  Hacker's  work  to  an  examination  of  school  education  and  of 
educational  technology  is  needed;  however,  in  the  meantime,  we  should  not 
lose  sight  of  the  influence  of  the  field  of  military  training  on  the  forms  and 
functions  of  educational  technology. 

Beyond  this  positioning,  educational  technology  reproduces  patriarchy 
because  it  inherits  and  reproduces  all  the  gender  biases  of  its  root  fields  and 
of  the  fields  of  "human  learning"  to  which  it  is  applied.  Biases  inherent  in 
learning  psychology,  in  the  various  technologies  of  media  and  materials,  in 
educational  measurement,  and  in  the  gendered  subjects  of  school  instruction 
interact  multiplicatively  as  they  are  brought  together  in  educational 
products  for  student  users.  A  brief  recap  of  some  of  these  biases  and  their 
effects  can  inform  our  unthinking.  Uncovering  the  ways  in  which  the  bodies 
of  content  of  subject  matter  instruction  are  gendered  has  been  the  major 
project  of  the  field  of  Women's  Studies.  Although  the  gender  biases  inherent 
in  the  canons  of  literature  and  the  study  of  history  have  been  examined 
most  fully,  the  work  of  Sandra  Harding  (cited  above)  and  of  many  other 
feminists  who  study  science  and/or  philosophy  has  uncovered  serious  bias  in 
the  sciences;  no  field  of  study  is  immune  to  the  presence  of  clearly  gendered 
content,  if  not  in  the  major  concepts  and  underlying  principles  of  the  field, 
then  in  its  illustrative  examples,  its  canonized  exemplars,  or  its  applications. 
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The  effects  of  gendered  content  upon  female  students  can  be  numerous:  the 
undervaluing  of  female  potential,  cognitive  dissonance,  and  the  confounding 
of  cognitive  and  affective  learning.  Consider,  for  eiample.  the  plight  of  the 
pubertal  young  woman  learning  in  biology  class  that  females  are  "unfinished 
males." 

The  gendering  oS  psychology  merits  special  consideration  because 
educational  technology  derives  principles  from  psychological  learning  theory 
and  implements  practices  based  upon  psychological  measurement.  The 
gendering  of  psychology  is  especially  pernicious  because  one  stated  intent  of 
the  field  is  to  study  and  understand  sex  differences.  As  early  as  1903  (when 
psychology  was  still  a  very  young  field),  the  female  psychologist  Helen 
Thompson  Woolley  argued  that  the  discipline  was  plagued  with  sei  bias, 
especially  in  the  area  of  sei  differences  research.  Since  that  time,  numerous 
scientists  have  expanded  on  Woolley's  observations.  Recent  feminist 
writings  on  the  topic  include  Corinne  Squires'  (1989)  analysis  which  reveals 
that  feminist  psychologists  have  found  psychology  to  be  non-  egalitarian  at 
all  levels.  Not  only  are  the  researchers  primarily  male,  and  the  subjects 
(who  are  the  objects)  of  study  historically  male,  but  also  the  methods  and 
theories  are  biased  in  gender  specific  ways.  For  eiample,  the  standard 
methods  of  psychological  investigation  vary  with  the  (prior  or  presumed) 
gender  correlates  of  a  trait  under  study;  characteristics  which  are 
considered  to  be  male  are  studied  using  high  status,  active  experimental 
methods,  while  those  considered  to  be  female  are  studied  using  lower  status, 
passive  methods  such  as  questionnaires  and  observation.  Thus  at  the  level 
of  investigation,  methods  and  gender  are  confounded.  Margrit  Eichler  (1988) 
ii;  s  identified  four  primary  problems  of  sexism  which  characterize 
pbvchological  (and  educational)  research:  androcentricity,  overgeneralization. 
geiider  insensitivity,  and  double  standards.  As  ..hown  in  her  analysis,  these 
problems  manifest  themselves  in  all  stages  of  the  research  process, 
be^ir.mng  with  the  choice  of  value  laden  topics  for  study  and  culminating  in 
the  exist  choice  of  interpretations  and  languages  used  in  the  reporting  of 
resf  its.  Eichler  observes  that  the  use  of  the  null  hypothesis,  and  the 
labelling  of  results  as  "significant"  only  when  the  hypothesis  is  rejected, 
creates  a  literature  of  difference.  As  a  consequence  of  this  research  practice, 
all  "significant"  results  in  the  psychological  study  of  gender  must  point  away 
from  samenesses  of  the  sexes.  As  Allison  Jaggar  (1987)  and  other  feminists 
have  argued,  the  overall  effect  of  sex  differences  research  (and  the  publicity 
surrounding  it)  is  to  rationalize  and  reproduce  inequality. 

Procedures  and  practices  of  Educational  Technology  tend  to  play  a 
considerable  role  in  the  reproduction  of  inequality.  The  educational  research 
on  which  educational  technology  relies  is  saturated  with  studies  sex 
differences,  so  much  so  that  when  studies  reveal  no  sex  differences  the 
researcher  is  almost  obliged  to  follow  up  with  some  sort  of  ATI  study  which 
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will  uncover  a  variable  whose  interaction  with  sex  can  "clarify"  and  "deepen" 
our  understanding  of  how  gender  operates  in  relation  to  the  original 
question.  Such  studies  form  the  starting  point  for  most  research  on 
instruction  and  the  development  of  instructional  procedures,  and  are  also 
used  in  relation  to  research  related  to  cognitive  psychology  and  expert 
systems.  After  a  difference  in  the  measured  performance  of  two  groups  is 
identified,  representatives  of  the  groups  are  studied  in  more  detail  with  the 
object  of  determining  characteristics  which  are  present  among  the  high 
performers  and  absent  ?mong  the  low  performers.  These  characteristics  are 
valorized  through  their  naming  and  their  designation  as  new  instructional 
objeaives.  Instruction  is  then  designed  to  help  the  low  achievers  reach 
these  new  objectives.  The  process  is  recursive  in  that  it  can  be  repeated 
with  new  objectives,  thus  dividing  the  instruction  into  smaller  and  smaller 
bites.  Whenever  the  high  achievers  are  predominantly  male  (as  they  are  in 
math  and  science,  especially)  one  effect  of  the  process  is  to  program  females 
to  behave  like  males.  Another  noteworthy  effect  of  the  procedure  is  that, 
because  instruction  is  modelled  on  the  spontaneous  behavior  of  the 
achievmg  male,  he  is  in  no  danger  of  losing  his  status  as  an  achiever;  if 
anything,  his  status  is  increased  with  the  number  of  objectives  that  he  can 
meet  effortlessly.  The  implications  for  the  female  are  entirely  different; 
already  caught  in  the  recursive  refinement  of  male  objectives,  her  responses 
to  a  "learning  situation"  go  unnoticed;  they  are  neither  categorized  and 
named,  nor  reified  as  objectives.  Whatever  her  reasons  for  so  responding, 
and  whatever  the  process  of  responding  may  have  contributed  to  her 
personal  learning  (building  her  cognitive  structure,  if  you  will)  remain  hers 
alone,  to  be  supplemented  with  the  new  "more  appropriate"  learning. 

In  summary,  educational  technology  as  used  by  students  is  thorougiily 
saturated  with  the  biases  of  its  root  disciplines  and  curricuiar  contexts. 
Gendered  subject  matter  has  different  meanings  for  females  than  for  males, 
and,  therefore,  is  likely  to  elicit  different  responses  and  different  strategies 
for  dealing  with  it.  When  these  differences  result  in  measured  differences  in 
performance  with  respect  to  standards  which  are  derived  from  the  male 
definition  of  appropriate  learning,  the  process  of  educational  technology 
intervenes  with  products  designed  to  instruct  females  on  th^*  details  of  male 
behavior  in  response  to  the  topic.  Initial  interpretations  and  learning  by 
females  go  unrecognized  in  the  educational  technology  system  (although 
they  remain  with  the  female).  These  gender  biases  and  effects  permeate  all 
of  education;  however,  by  eliminating  "noise"  such  as  indeterminacy, 
ambiguity,  and  inefficiency  from  the  instructional  process,  educational 
technology  has  the  potential  to  perpetuate  these  biases  in  their  purest  form. 

In  diferent  ways  the  contributions  of  the  teacher  user  and  the  female 
student  user  to  the  teaching-learning  process  are  denied  by  the  increasing 
use  of  educational  technology.  For  the  teacher  this  denial  takes  the  form  of 
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simultaneously  decreasing  her  authority  with  regard  to  the  content  of 
instruction,  increasing  the  level  of  ambiguity  in  her  day-to-day  activity,  and 
deprivmg  her  of  social  credit  for  the  increasingly  complex  job  which  she 
performs.  For  the  female  student,  denial  is  more  subtle.  Although  the 
learning  resources  that  she  uses  may  be  free  of  overt  sexism,  the  deeply 
gendered  characteristics  of  the  learning  environment  and  of  her  daily 
experience  work  together  to  deny  her  cognitive  autonomy. 

This  unthinking  of  the  effects  of  educational  technology  on  its  student 
and  teacher  users  is  but  one  beginning;  a  different  unthinking  or  a  further 
unthinking  might  bring  to  bear  feminist  research  on  women's  ways  of 
knowing  (Belenkey  et  al,  1986)  which  posits  a  new  and  female -grounded 
stage  theory  of  learning;  it  might  spring  from  a  feminist  analysis  of  the  the 
politics  of  education,  or  from  a  feminist  deconstruction  of  educational  equity. 
Additional  contextual  studies  of  educational  technology  should  examine 
effects  on  other  users,  as  well  as  the  positioning  and  meaning  of  educational 
technology  in  social,  cultural,  historical,  environmental,  and  other  contexts. 
Analyses  similar  to  the  user  analysis  of  this  study  should  consider  users 
from  different  races  and  classes  within  our  society.  In  short,  the  paragraphs 
above  comprise  but  one  of  the  many  ways  in  which  it  is  both  possible  and 
essential  for  feminists  to  unthink  educational  technology. 
ReThmking.  Energiting.  Trafisforming 

Feminist  unthinking,  like  post-modern  deconstruaion.  shatters  myths 
of  value  neutrality  and  frees  us  from  the  tyrrany  of  absolutism;  it  allows  the 
rethinking  and  reconstruction  of  texts  and  technologies  from  new  value 
bases.  Feminist  unthinking  is  not  a  form  of  Luddism.  but  begins  with  the 
premise  that  technologies  are  neither  wholly  good  nor  wholly  bad. 
Technologies  are  products  of  the  societies  from  which  they  emerge,  and  our 
society  is  patriarchal;  it  is  no  surprise,  therefore,  that  feminist  unthinking 
begms  to  unravel  ways  in  which  educational  technology  is  deeply  gendered 
and  massively  sexist.  The  question  for  feminist  educators  is  whether 
unthinking  its  myths  reveals  educational  technology  to  be  so  heavily 
valenced  toward  the  masculine  that  it  can  have  no  place  in  a  feminist 
society;  or.  on  the  other  hand,  does  the  unthinking  suggest  ways  in  which  it 
can  be  rethought  as  a  more  feminist  technology. 

The  unthinking  of  the  previous  pages  evokes  several  questions  which 
might  guide  feminist  rethinking  of  educational  technology,  among  them  the 
following:  Can  the  monolitnic  hierarchical  structure  of  "educational 
technology"  be  rethought  as  a  group  of  diverse  "technologies  of  education"? 
What  technologies  of  education  would  teachers  invent?  And  how  would  they 
use  them?  How  can  we  re-invent  a  technology  of  education  based  upon 
some  of  the  new  feminist  research  on  the  psychology  of  women?  How  can 
the  power  and  flexibility  of  the  computer  be  used  to  malie  the  experiences, 
writings,  and  products  of  women  (as  well  as  men)  available  to  students  as 
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valuable  learning  resources?  How  can  multiple  technologies  be  used  in  the 
education  of  students  with  multiple  interpretations  of  texts,  experiences,  and 
reality?  How  can  educational  technology  foster  students'  understanding  of 
the  interrelatedness  of  perceptions  and  phenomena,  and  of  indeterminacy? 
What  is  the  place  of  technology  in  a  nurturing  educational  environment? 
How  can  technologies  be  used  in  ways  that  do  not  deny  thf  realities  and 
needs  of  the  individuals  using  them?  To  take  these  questions  seriously  is  to 
find  energy  and  to  begin  to  transform  educational  technology. 
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Effects  of  Students'  Prior  Knowledge  and  Presentation  Mode 
on  Achievement  (Visual/Verbal  Testing)  of 
Different  Educational  Objectives 

Abstract 

The  purposes  of  this  study  w^-re  to  determine  (a)  the  effectiveness  with  which 
different  types  of  rehearsal  strategies  complementing  visualized  instruction  facilitates 
student  achievement,  (b)  the  effect  of  different  instructional  treatment  on  students' 
processing  different  prior  knowledge  levels  ;And  (c)  whether  verbal  and  visual  tests  are 
equally  effective  in  retrieving  information  acquired  from  visualized  instruction.  One 
hundred  twenty  students  enrolled  at  The  Pennsylvania  State  University  were  randomly 
assigned  to  four  treatment  groups  which  determined  both  the  method  of  instruction  and 
testing  mode  they  received.  Students  received  their  respective  instructional  presentations 
and  criterion  tests  in  one  session.  Analyses  indicated  that:  (a)  students  possessing  different 
entry  ievels  of  prior  knowledge  profit  differentially  from  the  different  treatments;  (b) 
differences  in  achievement  potential  among  students  with  prior  knowledge  levels  can  be 
reduced  in  terms  of  information  acquisition  by  providing  treatments  with  elaborate 
rehearsal  activity;  (c)  identical  treatments  are  not  equally  effective  in  promoting  student 
achievement  for  different  types  of  educational  objectives  and  (d)  visual  testing  is  a  viable 
strategy  for  retrie>ang  information  acquired  by  students  receiving  visualized  instruction. 


Introduction 

Rehearsal 

Considerable  research  (Paivo,  1971,  Tulving,  1976,  Dwyer,  197S,  1987)  has  indicated  that 
visualized  instruction  alone  does  not  always  maximize  student  learning,  Gagne  (1977) 
indicated  that  learning  is  a  highly  idiosyncratic  event,  and  depends  very  much  on  the  nature 
of  the  learner  -  particularly  on  his  prior  learning*  In  order  for  learning  to  be  effective  at 
any  level,  the  learners  must  be  acuve  (Bork,  1979;  Bransford,  1979;  Travers,  1970),  In  this 
regard,  Murray  and  Mosberg  (1982)  have  indicated  that  the  longer  an  individual  can  be  in 
rehearsal  type  activities  with  the  content  material,  the  greater  the  probability  that  learning 
will  occur  and  be  retained.  Mental  activity  on  the  part  of  learners  is  essential  for  learning 
to  occur.  This  activity  includes  the  selection  and  perception  of  stimuli;  encoding  of  new 
stimuli,  and  the  retrieval  of  prior  knowledge  for  use  in  combination  with  the  new  stimuli  for 
imaging,  comparison,  analysis,  synthesis,  and  problem-solving.  Varied  forms  of  rehearsal, 
while  focusing  attention,  allow  time  for  incoming  information  to  remain  in  short-term 
memory  long  enough  to  be  elaborated  upon  and  encoded  for  long-term  memory  (Anderson, 
1980;  Dushkin,  1970;  Lindsay  &  Norman,  1972;  Murray  &  Mosberg,  1982).  Lindsay  and  Norman 
(1972)  have  argued  that  the  longer  an  item  is  maintained  in  short-term  memory  by  rehearsal, 
the  greater  the  probability  that  it  will  be  transmitted  into  long-term  memory  and  be 
retained. 

In  general,  rehearsal  can  be  considered  to  be  any  mathemagenic  activity  that  can 
serve  the  learner  in  several  ways  including  motivating  and  promoting  appropriate  mental 
activity.  Additionally,  different  rehearsal  strategies  differ  in  intensity  of  learner 
involvement  and,  thus,  may  have  differential  effects  on  specific  learning  outcomes.  It  may 
be  that  the  effectiveness  inherent  in  rehearsal  is  that  it  allows  the  learner  to  process 
information  simultaneously  on  several  levels.  If  this  is  the  case,  then  it  may  also  follow  that 
optimum  intensity  of  rehearsal  activity  (actual  overt  interaction  with  the  content  material) 
may  be  directly  related  to  the  level  of  learning  to  be  achieved-the  more  complex  learning 
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requiring  the  most  intense  or  involved  type  of  rehearsal  activity.  For  example,  covert 
rehearsal  generally  requires  minimal  information  processing  activity  on  the  part  of  the 
learner.  Examples  of  covert  rehearsal  include  reading  prose  passages,  reading  summary 
statements,  reading  questions  and  answering  them  mentally  before  checking  with  a  given 
answer,  and  following  mentally  the  completed  solution  of  a  problenri»  Researchers  have 
found  that  reading  correct  statements  or  correct  answers  does  not  always  provide  for  a  level 
of  menta!  processing  that  results  in  increased  understanding  (Anderson,  Goldberg,  &  Middle, 
1971;  Bransford,  1979).  Overt  rehearsal,  by  providing  physical  activity  in  which  the  learner  is 
required  to  interact  with  the  content  material,  ensures  that  he/she  attends  to  the 
information  and  spends  more  time  interacting  with  and  encoding  the  infornnation. 

Visual  Testing 

The  transmittal  and  retrieval  of  intornnation  are  primary  concerns  in  any 
instructional/training  environment.  In  considering  factors  that  might  e;nhance  information 
acquisition  and  retrieval,  Tulving  and  Thomson  (1973)  have  proposed  the  encoding  specificity 
principle— that  recognition  memory  is  better  if  the  cues  used  in  the  original 
instructional/acquisition  environment  are  used  in  the  testing  retrieval  environment. 
Similarly,  Battig  (1979)  and  Nitsch  (1977),  in  adhering  to  the  encoding  specificity  principle, 
have  indicated  that  any  change  in  the  retrieval  environnrient  from  that  which  occurred  in  the 
original  learning  environment  produces  marked  decrements  in  learner  performance. 

Support  for  the  use  of  visual  test  items  that  employ  visuals  of  the  same  type  as  those 
employed  in  the  instructional  has  surfaced  regularly  in  the  research  literature  in  the  form  of 
hypotheses  and  theories;  for  example,  the  sign-similarity  hypotheses  (Carpenter,  1953),  cue 
summation  theory  (Severin,  1967),  stimulus-generation  theory  (Hartman,  1961),  encoding 
spedficity  principle  (Tulving  and  Thomson,  1973),  and  transfer^appropriate  processing 
principle  (Morris.  Bransford,  and  Franks,  1977).  In  this  regard  Lindsay  and  Norman  (1977,  p. 
337)  have  stated  that  in  the  teaching-learning  environment,  "the  problem  in  learning  new 
information  is  not  getting  the  information  into  memory;  it  is  making  sure  that  it  will  be 
found  later  when  it  is  needed."  Bransford  (1979),  Tulving  (1979),  and  Tulving  and  Osier  (1968) 
have  indicated  that  the  accuracy  with  which  information  is  retrieved  is  related  to  the 
degree  of  elaborateness  of  the  encoding  which  occurred  during  the  rehearsal  activity. 
Similarly,  Tulving  and  Thompson  (1973),  in  discussing  the  retrieval  of  encoded  information, 
maintain  that  memory  is  better  if  the  cues  used  in  the  original  information  activity  are 
employed  in  the  retrieval  situation. 

Consequently,  information  retention  level  is  assumed  to  be  a  direct  function  of  the 
encoding  occurring  at  the  presentation  stage  and  the  degree  to  which  the  retrieval 
environment  recapitulates  this  encoding  (Battig,  1979;  Tulving,  1979).  The  implications  of 
this  position  would  imply  that  in  instructional  situations  where  visualization  was  utilized  in 
the  encoding  process  and  was  not  used  in  the  retrieval  (decoding)  process,  learner 
performance  measures  would  yield  groas  underestimates,  if  not  distortions,  with  respect  to 
what  and  how  much  information  had  been  originally  acquired.  This  conceptualization 
suggests  that  information  retrieval  is  a  very  specific  process,  easily  disrupted.  Since  the 
features  of  the  original  learning  cues  are  processed  during  a  test,  any  reduction  in  the 
individual  distinctiveness  of  the  cues  themselves  should  produce  concomitant  reductions  in 
recall  (Nelson,  1979). 

Problem  Statement 

The  purpose  of  this  study  was  to  investigate  the  instructional  effectiveness  of 
integrating  rehearsal  activity  into  visually  complemented  prose  instruction.    Within  this 
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context  the  instructional  effectiveness  of  both  overt  and  covert  rehearsal  activity  was 
examined.  Additionally,  the  study  examined  the  students^  levM  of  prior  knowledge  on 
learning  and  the  effect  of  visual/verbal  testing.  Specifically,  the  purpose  oi  inis  siuay  was 
to  determine  whether  (a)  students  possessing  different  prior  knowledge  levels  profit 
differentially  from  different  instructional  treatments,  and  (b)  visual  testing  is  a  viable 
strategy  for  retrieving  information  acquired  by  students  receiving  visualized  instruction. 

Verbal  and  Visual  Tests  (Crixerion  Measures) 

Students  in  each  instructional  group  participated  in  their  respective  treatments 
followed  immediately  by  a  drawing  test.  Each  student  then  took  three  separate  20-item 
multiple-choice  criterion  tests,  either  in  the  verbal  format  or  visual  format.  Scores  in  these 
three  individual  criterion  tests  (identification,  terminology,  and  comprehension)  were 
combined  into  a  60-item  composite  test  score. 

The  three  multiple-choice  tests  (verbal  iormat)  used  in  this  investigation  were 
developed  by  Dwyer  (1972).  Additional  revisions  were  made  to  selected  multiple-choice 
questions  to  further  eliminate  the  ambiguity  of  specific  distractors  and  to  attempt  to 
prevent  a  specific  question  and  its  distractors  from  claeing  another  answer  (Dwyer,  1985- 
1986).  The  format  of  each  of  the  60  multiple-choice  items  consisted  of  a  typical  verbrl  stem 
and  verbal  response  options.  The  following  description  of  the  criterion  tests,  adapted  irom 
Dwyer  (1978,  pp.  'f5-^7)  illustrates  the  kinds  of  educational  objectives  assessed  in  this  study. 

Drawing  Test  The  objective  of  the  drawing  test  was  to  evaluate  student  ability  to  construct 
and/or  reproduce  items  in  their  appropriate  context.  The  drawi'.g  test  (20  items)  provided 
the  students  with  a  numbered  list  of  terms  corresponding  to  the  parts  of  the  heart  discussed 
in  the  instructional  presentation.  The  students  were  required  to  draw  a  representative 
diagram  of  the  heart  and  place  the  numbers  of  the  listed  parts  in  their  respective  positions. 
For  this  test  the  emphasis  was  on  the  correct  positioning  of  the  verbal  symbols  with  respect 
to  one  another  and  in  respect  to  their  concrete  referents. 

Identification  Test  (Verbal  and  Visual  Format)  The  objective  of  the  identification  test  was 
to  evaluate  student  ability  to  identify  parts  or  positions  of  an  object.  This  multiple-choice 
test  (20  items)  required  students  to  identify  the  numbered  parts  on  a  detailed  drawing  of  a 
heart.  Each  part  of  the  heart,  which  had  been  discussed  in  the  presentation  was  numbered 
on  a  drawing.  The  objective  of  this  test  was  to  measure  the  ability  of  the  student  to  use 
visual  cues  to  discriminate  one  structure  of  the  heart  from  another  and  to  associate  specific 
parts  of  the  heart  with  their  proper  names. 

Terminology  Test  This  test  :onsisted  of  20  multiple-choice  items  designed  to  measure 
knowledge  of  specific  facts,  terms,  and  definitions.  The  objectives  measured  by  this  type  of 
test  are  appropriate  to  all  content  areas  that  have  an  understanding  of  the  basic  elements  as 
a  prerequisite  to  the  learning  of  concepts,  rules,  and  principles. 

Comprehension  Test  The  comprehension  test  consisted  of  20  multiple-choice  items.  Given 
the  location  of  certain  parts  of  the  heart  at  a  particular  moment  of  its  functioning,  the 
student  was  asked  to  determine  the  position  of  other  specific  parts  of  the  heart  at  the  same 
time.  This  test  required  that  the  students  have  a  thorough  understanding  of  the  heart,  its 
parts,  its  internal  functioning,  and  the  simultaneous  processes  occurring  during  the  systolic 
and  diastolic  phases.  The  comprehension  test  was  designed  to  measure  a  type  of 
understanding  in  which  the  individual  can  use  the  information  being  received  to  explain  some 
other  phenomenon. 
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Composite  Test  Score  The  items  contained  in  three  of  the  four  individual  criterion  tests 
(identification,  terminology,  and  comprehension)  were  combined  into  a  69-item  composite 
test  score.  The  purpose  was  to  measure  the  total  achievement  of  the  varied  levels  of 
objectives  presented  in  the  instructional  unit. 

In  designing  the  visual  test  tormats  for  each  criterion  test,  the  visual  tests  developed 
by  De  Melo  (1980)  were  used  as  a  guide.  De  Melons  visual  tests  utilized  four  or  five  drawings 
as  distractors  for  each  question.  The  revised  version  of  the  visual  form  of  the  criterion 
tests  utilized  only  one  drawing  with  four  or  five  letter  labels  in  all  items  in  which  it  was 
possible  to  do  so  while  maintaining  clarity  and  correspondence  to  the  verbal  test  items  (see 
Figure  I;.  However,  two  items  in  the  terminology  test  and  all  items  in  the  comprehension 
test  required  four  drav/ings.  The  item  stems  of  both  the  verbal  and  visual  test  questions 
were  verbal  and  asked  the  same  question-  In  addition,  the  visual  distractors  in  the  visual 
tests  corresponded  to  the  verbal  distractor**  in  the  verbal  tests  \z  Uosely  as  was  reasonable. 
The  description  of  the  verbal  tests  given  previously  also  describes  i."  visual  tests. 


Figure  1  About  Here 


Instructional  Treatments 

Each  of  the  four  instructional  treatments  in  this  study  contained  the  same 
instructional  script,  visuals,  terminology  labels,  and  arrows.  The  treatments  differed  only  in 
the  degree  of  rehearsal  employed  (covert  or  overt)  and  the  type  of  testing  employed  (verbal 
or  visual).  Figure  2  (Plate  1)  illustrates  a  sample  frame  received  by  students  in  Treatments  1 
(Reading  Summaries  -  Verbal  Test)  and  Treatment  2  (Reading  Summarie.  -  Visual  Test). 
Students  ir.  both  treatments  received  the  same  instruction;  however,  students  in  Treatment  1 
received  verbal  tests  while  students  in  Treatment  2  received  visual  tests.  The  summary 
statements  at  the  end  of  each  page  required  a  minimal  amount  of  covert  rehearsal  on  the 
part  of  the  students  and  did  not  review  all  the  information  in  Ihe  instructional  script.  The 
summary  statements  were  designed  to  provide  the  students  with  the  opportunity  for  mental 
review  of  the  instructional  content. 

Students  in  Treatments  3  (Overt  Visual  Rehearsal-Verbal  Test)  and  Treatment  ^  (Overt 
Visual  Rehearsal-Visual  Test)  received  identical  instruction  (Figure  2,  Plate  2);  however, 
students  in  Treatment  3  received  verbal  tests  while  students  in  Treatment  ^  received  visual 
tests.  These  treatments  required  that  students  shade  with  colored  pencils  the  specified 
parts  and  functions  of  the  heart.  The  parts  and  functions  to  be  shaded  corresponded  to,  as 
closely  as  possible,  the  information  summarized  in  Treatments  1  and  2.  The  colors  used  were 
selected  to  avoid  a  sense  of  color  coding:  black,  green,  yellow  and  red.  Red  was  used  in  two 
circumstances  in  which  four  colors  were  required  on  a  page;  association  of  the  red  color 
with  oxygen-rich  biood  was  avoided. 

Instruction  was  presented  to  students  in  booklet  format  and  students  were  permitted 
to  spend  as  much  time  as  they  needed  to  interact  with  the  instructional  content  and  to 
complete  the  criterion  tests. 


Figure  2  About  Here 
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Design  and  Analyses 


One  hundred  twenty  undergraduate  students  enrolled  at  The  Pennsylvania  State 
University  participated  in'this  study.  A  pretest  consisting  of  36  items  on  general  content  in 
physiology  (Dwyer,  1972)  was  utilized  in  this  study.  The  pretest  was  used  to  determine 
students*  prior  knowledge  level  regarding  human  physiology.  Scores  on  the  pretest  were 
arranged  in  decending  order  from  highest  to  lowest.  The  top  ^0  scores  represented  high 
prior  knowledge  (M=27.3),  the  next  W  medium  prior  knowledge  (M=21.8)  and  the  bottom  ^0 
low  prior  knowledge  (M=17.1).  The  Formula  21  reliability  of  the  physiology  pretest  was  .8^ 
and  its  correlation  with  the  total  composite  tests  was  .56.  Students  in  each  of  the  three 
prior  knowledge  levels  were  then  randomly  assigned  into  one  of  the  prior  treatment  groups. 
The  independent  variables  manipulated  were  levels  of  prior  knowledge,  level  of  rehearsal 
strategy  (covert/overt)  and  test  mode  (verbal/visual).  The  dependent  variables  were  (a) 
performance  on  the  visual  and  verbal  versions  of  the  individual  criterion  tests— terminology, 
identification,  and  comprehension,  (b)  performance  on  the  composite  test,  and  (c) 
performance  on  the  drawing  test. 

Alpha  was  set  at  the  .05  level  for  each  analysis  of  variance.  Where  significance  was 
found  to  exist  Tuhey's  Wholly  Significant  Difference  (W5D)  for  comparison  among  the  means 
was  utilized. 

Results 

Table  1  presents  the  means,  variances  and  reliability  coefficients  on  the  criterion 
tests.  Analyses  of  variance  indicated  that  significance  differences  existed  on  the  drawing 
(F=9.62,  d.=3/108,  p.  <.05),  identification  (F=19.86,  d,=3/108,  p.  <.05),  terminology  (F=22.63, 
d -=3/108,  p.  <.05),  comprehension  (F=3^.l5,  dj=3/102r,  p.  <  .05)  and  composite  tests  (F=31.06, 
d.-3.108,  p.  <.05).  Figure  3  illustrates  the  comparison  of  mean  achievement 


Table  1  About  Here 


Figure  3  About  Here 


scores  for  each  prior  knowledge  level  on  each  of  the  criterion  measures.  These  comparisons 
generally  indicate  that  Treatment  ^,  the  shading  on  pictures  (overt  rehearsal):  visual 
testing,  was  the  most  effective  in  facilitating  information  acquisition.  Table  2  shows  where 
significant  differences  occurred  in  achievement  among  students  possessing  different  prior 
knowledge  levels  (High,  Medium,  Low)  as  the  five  criterion  measures.  The  blank  areas 
indicate  where  significant  differences  in  achievement  did  not  occur.  These  results  illustrate 
the  fact  that  the  low  prior  knowledge  group  is  the  group  which  is  most  positively  influenced 
by  the  different  instructional  strategies.  Additionally,  Treatment  ^  was  the  instructional 
format  which  was  most  instrumental  in  reducing  the  effect  of  differences  among  students  in 
the  prior  knowledge  levels. 
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Tabic  2  About  Here 


Table  3  illustrates  the  results  of  comparing  studtnt  performance  receiving  similar 
instruction  but  receiving  tests  in  the  different  evalution  modes.  Insignificant  differences 
were  found  to  exist  between  students  who  received  the  reading  summaries  treatment  which 
were  evaluated  by  both  visual  and  verbal  tests.  However,  in  evaluating  the  differences 
between  Treatments  3  and  ^,  Treatment  ^  the  Shading  on  Drawing  (Overt  Rehearsal)  Visual 
Test  was  tound  to  be  significantly  more  effective  than  Treatment  3  on  all  the  criterion 
measures. 


Table  3  About  Here 


Discussion 

Considerable  research  on  the  design  and  use  of  visuals  has  shown  that  visualization  can 
significantly  improve  students'  learning  from  prose  instruction  (De  Melo,  1980;  Dwyer,  1978; 
Dwyer  &  Parkhurst,  1982;  Levie  &  Levie,  1975,  1971).  The  review  of  literature  for  this  study 
indicated  that  rehearsal  strategies  and  the  visual  test  mode  are  important 
instructionalvariables.  The  present  study  went  one  step  beyond  merely  establishing  the 
importance  of  visualization  but  attempted  to  determine  (a)  the  effect  of  different  rehearsal 
strategies  used  to  complete  visualized  prose  instruction,  (b)  the  effect  visual  testing  in 
retrieving  information  and  (c)  the  etfect  that  different  instructional  strategies  have  on 
students  possessing  different  prior  knowledge  levels. 

Results  of  this  study  indicate  that  all  types  of  rehearsal  strategies  are  not  equally 
effective  in  facilitating  student  achievement  of  different  educational  objectives  (Figure  3). 
The  general  trend  also  seems  to  indicate  that  overt  rehearsal  is  more  effective  than  covert 
rehearsal  in  facilitating  student  achievement.  Additionally,  it  was  found  that  within  the 
overt  rehearsal  treatments  when  significant  differences  were  found  to  exist  students  who 
received  the  visual  test  mode  achieved  significantly  higher  scores  than  did  students  who 
received  the  verbal  tests.  The  higher  scores  on  the  visual  tests  by  students  in  Treatment  ^ 
may  be  explained  by  the  encoding  specificity  principle  (Battig,  1979;  Nitsch,  1977;  Tulving, 
1979)  since  the  visual  test  situation  in  this  study  closely  matched  the  learning  situation;  the 
visuals  employed  in  the  test  situation  provided  the  critical  cues  needed  by  students  to 
retrieve  the  encoded  information  (Bransford,  1979).  These  results  are  also  congruent  with 
the  sign-similarity  hypothesis  (Carpenter,  1953),  the  stimulus-generalization  hypothej:is 
(Hartman,  1961),  the  cue  summation  theory  (Severin,  1967),  and  the  transfer-appropriate 
principle  (Morris,  Bransford,  &  Franks,  1977). 

In  comparing  the  results  ol  the  verbal-visual  mode  of  testing  on  the  different  criterion 
measures  (Table  3)  insignificant  differences  were  found  to  exist  between  students  in 
Treatments  1  and  2.  Two  possible  explanations  may  be  proposed  for  this  finding:  (a)  the 
reading  summaries  treatments  (covert  rehearsal)  apparently  did  not  provide  enough 
maintenance  rehearsal  to  allow  for  additional  elaborative  rehearsal  that  would  encode  more 
information  from  short-term  memory  into  long-term  memory,  and  (b)  performance  on  the 
visual  test  may  have  been  influenced  by  the  fact  that  visual  tests  are  rather  unfamiliar  to 
most  students  and  their  performance  on  then  suffered  accordingly. 
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In  examining  the  effect  of  the  verbal-visual  testing  mode  (Treatment  1  vs.  Treatment  2 
and  Treatment  3  vs.  Treatment  Table  3)  insignificant  differences  were  found  to  exist 
between  Treatments  1  and  2  on  all  criterion  measures;  however,  Treatment  ^,  the  visual  test 
format  was  found  to  be  significantly  more  effective  than  its  counterpart  the  verbal  testing 
format  on  all  criterion  measures.  Apparently,  the  visual  rehearsal  which  required  that 
students  overtly  interact  with  the  instructional  content  by  requiring  them  to  color  and  draw 
on  the  visuals  functioned  to  provide  appropriate  conditions  for  encoding;  thus,  it  was 
possible  for  students  to  retrieve  more  information  on  the  visual  criterion  tests.  This  finding 
seems  to  support  the  encoding  specificity  principle  (Tulving  and  Thomas,  1973)  which 
contends  that  recognition  memory  is  better  if  the  cues  used  in  the  original 
instructional/acquisition  environment  are  used  in  the  testing/retrieval  environment. 
Additional  related  research  tends  to  support  this  position  (Nitsch,  1977;  Battig,  1979;  Morris, 
Bransford  &  Franks,  1977;  Dwyer  &  De  Melo,  198^;  Dwyer  &  Dwyer,  1985). 

In  examining  the  effect  of  the  different  treatments  on  students  possessing  different 
levels  of  prior  knowledge  (Table  2),  the  results  indicated  that  Treatments  1  and  2  had 
moderate  effects  in  reducing  prior  knowledge  differences  on  the  criterion  tests  and  when 
they  did  occur  differences  between  the  medium  and  low  prior  knowledge  levels  were 
effected.  Treatment  3  was  exfective  in  reducing  all  differences  between  the  low  and 
medium  prior  knowledge  levels.  On  three  criterion  measures  differences  between  the  low 
and  medium  prior  knowledge  levels  were  reduced.  Treatment  ^,  shading  on  pictures,  visual 
test,  influenced  student  performance  in  all  the  criterion  tests  dramatically  by  reducing 
performance  differences  attributed  to  levels  of  prior  knowledge.  The  success  of  this 
treatment  may  be  an  indication  of  the  dual  coding  which  occurred  (Paivio,  1971)— verbal 
encoding  from  the  interaction  with  the  verbal  text  and  visual  encoding  from  the  repeated 
interaction  with  the  visualized  content.  This  repeated  interaction  resulted  in  a  greater 
ar?.ount  of  information  being  encoded  relative  to  the  functions  of  ti  e  heart  a. id  the 
relationships  between  the  parts  of  the  heart.  It  may  be  that  students  by  their  sustained  and 
repeated  interaction  with  the  visuals  in  Treatment  ^  maintained  more  information  in  short- 
term  memory  for  longer  periods  of  time  which  allowed  for  more  elaboration  (by  students  of 
all  levels)  and  storage  in  long-term  memory  (Craik  &  Watkins,  1973).  This  is  consistent  with 
Travers'  (1982)  statement  that  short-term  memory  is  where  information  is  organized  and 
prepared  for  long-term  memory.  The  achievement  of  higher  scores  did  require  a  higher 
level  of  processing,  more  elaborations,  more  "links,"  more  reflection,  etc.  In  this  regard, 
the  data  indicated  that  more  information  was  encoded  in  the  visual  rehearsal  situation  as 
tested  by  the  visual  criterion  tests. 

Summary 

The  results  of  this  study  reveal  that  all  types  of  rehearsal  strategies  used  to 
complement  visualized  instruction  are  not  equally  effective  in  facilitating  student 
achievement  of  different  educational  objectives  and  that  visual  testing  is  an  important 
instructional  variable  for  facilitating  the  retrieval  of  optimum  amounts  of  acquired 
information.  The  finding  that  is  is  possible  to  develop  an  instructional-evaluation  strategy 
(i.e.  Treatment  ^)  which  can  reduce  differences  attributed  to  prior  knowledge  levels  is 
significant  for  future  instructional  design  and  development  activities.  However,  it  is 
important  that  these  findings  be  reflected  with  larger  and  more  varied  audiences  before 
these  findings  are  generalized  too  broadly. 
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Plite  1— Simple  quesUons  from  the  IdenUnctUon  twt  (visual  formii). 


lortk  vilve 


left  auricle 


Plate  2-S«mpie  questions  from  the  terminolouf  test  (visual  formiO. 


The  chamber  of  the  heart  that 
pumps  oxygenated  blood  to 
all  parts  of  the  body: 


The  part(s)  of  the  hein  that 
control(s)  its  contracuon  and 
relaxauon: 


Plate  3.  Sample  questions  from  the  comprehension  test  (risuii  formii). 

The  parts  of  the  heart  though  which  blood  is  being  forced 
dunng  the  second  contraction  of  the  systolic  phase: 


Figure  l-Sample  questions  from  the  tesU  In  the  visual  format. 
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Figure  2.      Samples  of  the  Rehearsal  Strategies 
>late  1.     Sample  page  from  Treatment  1     (Reading  Summaries:     covert  rehearsal) 


Pulmooary 
veins 


Mitral  YftlYC 


Tendons 


Returning  from  the  lungs,  the  blocd  enters  the  heart 
through  four  pulmonary  veins  and  collecu  in  the  left  auricle; 
these  vein  openings,  likr  the  vena  cava,  have  no  valves.  The 
left  auncle  contracts  when  it  is  full,  squeezing  blood  through 
the  mitral  valve  into  the  left  ventncle. 

The  mitral  valve,  located  between  the  left  auricle  and  the 
left  ventricle,  is  similar  in  construction  to  the  Sricuspid  valve. 
As  the  left  ventncle  coniracu  simuluneously  with  its  niate. 
the  nght  ventncle.  it  forces  blood  behind  the  flaps  of  the 
valve,  thereby  closing  the  passageway  back  to  the  left  aun- 
cle. Like  the  tncuspid  valve,  the  ends  of  the  mitral  valve 
have  flaps  that  are  anchored  to  the  floor  of  the  left  ventricle 
by  slender  tendons. 

(10)  As  the  blood  returns  to  the  heart  from  the  lungs,  it 
enters  the  heart  through  pulmonary  veins  and  is 
deposited  in  the  left  auncle. 

( 1 1)  The  mitral  valve  is  located  between  the  '     auricle  and 
the  left  ventncle. 


Plate  2.     Sample  page  from  Treatment  3      (Shading  on  Drawing:     overt  rehearsal) 
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-  Mltrml  valve 


Tendons 

Returning  from  the  lungs,  the  blood  enters  the  heart 
through  four  pulmonary  veins  and  collects  in  the  left  auncle: 
these  vein  openings,  like  the  vena  cava,  have  no  valves.  The 
left  auricle  contracts  when  it  is  full,  squeezing  blood  through 
the  mitral  valve  i  'o  the  left  vcntnde. 

The  mitral  valve,  located  between  the  left  auride  and  the 
left  ventricle,  is  similar  in  construaion  to  the  tncuspid  valve. 
As  the  left  ventncle  contracts  simultaneously  with  iu  mate, 
the  right  ventricle,  it  forces  blood  behind  the  flaps  of  the 
valve,  thereby  closing  the  passageway  back  to  the  left  auri- 
cle. Like  the  tricuspid  valve,  the  ends  of  the  mitral  valve 
have  flaps  that  are  anchored  to  the  floor  of  the  left  ventnde 
by  slender  tendons. 

(15)  Color  the  pulmonary  veins  black. 

(16)  Color  the  left  auricle  green. 

(17)  Color  the  mitral  valve  yellow. 
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Figure  3.  Comparison  of  Mean  Achievement  on  the  Criterion  Tests 
Plate  1.  Terminology  Test 


zu.uu 

18.00 

16.00 

Mean 

achieve- 

14.00 

ment 

12.00 

10.00 

8.00 

2  3 
Treatments 


20.00 
18.00 
16.00 

Mean 

achieve-  14.00 
ment 

12.00 
10.00 
8.00 


Plate  2.  Drawing  Test 


1  2  3 

Treatments 


Plate  3.  Identification  Test 


Plate  4.  Comprehension  Test 
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Mean 
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2  3  4 

Treatments 


Plates.  Composite  Test 


60.00 

55.00 

Mean 

50.00 

achieve- 

45.00 

ment 

40.00 

35.00 

30.00 

2  3 
Treatments 


20.00 
18.00 

Mean  ^^"^ 
achieve-  14.00 

""^^^  12.00 

10.00 

8.00 


Legend 

High  prior  knowledge  level 

Medium  prior  knowledge  level 

I  ow  prior  knowledge  level 

Treatments 

1.  Reading  summaries  (coven  rehearsal):  verbal  tests 

2.  Reading  summaries  (covert  rehearsal:  visual  tests 

3.  Shading  on  pictures  (overt  rehearsal):  verbal  tests 

4.  Shading  on  pictures  (overt  rehearsal):  visual  tests 
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Table  1.  Means,  Variance  Estimates  and  Reliability  Coefficients  on  the  Criteria  Tests 

Criterion  Tests 

TREATMENTS  Termin-    Identi-  Compre- 

Drawing     ology        fication    hension  Composite 


lie  m5 

(on  \ 

1  I  ccl  LI  lid  1  L  I*  iXcclUllig 

n  Q7 

in  Cf\ 

Summons  (Covert  Re- 

Variance 

11.60 

7.82 

7.18 

9.36 

56.18 

licctrbctl*    Vcrucll  1651 

7Q 

71 
•  / 1 

•oi 

111 

97 

Treatment  2.  Reading 

Mean 

12.20 

l'f.63 

12.30 

39.13 

Summons  (Covert  Re- 

Variance 

11.60 

7.82 

7.18 

9.36 

56.18 

hearsal:  Visual  Test 

K-R  21 

.81 

.63 

.77 

.67 

.72 

Treatment  3.  Shading  on 

Mean 

15.93 

15.57 

16.93 

lif.l3 

'f6.60 

Pictures  (Overt  Re- 

Variance 

11.60 

7.82 

7.18 

9.36 

56.18 

hearsal):  Verbal  Test 

K-R  21 

.77 

.83 

.73 

.79 

.83 

Treatment  ^.  Shading  on 

Mean 

18.27 

17.70 

18.97 

18.20 

5't.80 

Pictures  (Overt  Re- 

Variance 

11.60 

7.82 

7.18 

9.36 

56.18 

hearsal):  Visual  Test 

K-K  21 

.83 

.80 

.79 

.69 

.78 
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Table  2:    Effect  of  Instructional  Treatment  on  Prior  Knowledge  Level  on 

Each  Criterion  Test 


Instructional  Treatments 


Criterion  Tests 


Treatment 
Reading 
Summaries 
(Covert 
Rehearsal) : 
Verbal  Test 


1: 


Treatment  2: 
Reading 
Summaries 
(Covert 
Rehearsal : 
Visual  Test 


Treatment  3: 
Shading  on 
Pictures 
(Overt 
Rehearsal ) : 
Verbal  Test 


Treatment  4: 
Shading  on 
Pictures 
(Overt 
Rehearsal) : 
Visual  Test 


Pretest  1>2  1>2  1>2  1>2 

1>3  1>3  1>3  1>3 

2>3  2>3  2>3  2>3 

Drawing  Test  1>2  1>2  1>2 

1>3  1>3  1>3 


Identification  1>2  1>2  1>2 

Test  l>3  1>3 

2>3  2>3 


Terminology  1>2  1>2 

Test  1>3  1>3  1>2 


Comprehension  1>2  1>2  l>2  1>2 

Test  1>3  1>3  1>3 

2>3 


Composite  Test  1>2  1>2  1>2 

1>3  1>3 
2>3 


1  =  High  prior  knowledge  level 

2  -  Medium  prior  knowledge  level 

3  =  Low  prior  knowledge  level 
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Table  3:     Comparison  of  Visual-Verbal  Testing  on  the  Criterion  Measures 


.05  Level 

Treatment  1. 
Treatment  2. 
Treatment  3 . 
Treatment  4: 


T 

R 

Drawing 

Terminology 

Identification 

E 

Test 

Test 

Test 

A 

T 

1=2 

1=2 

1  =  2 

M 

E 

4>3 

4>3 

4>3 

N 

T 

S 

Comprehension 
Test 

1»2 

4>3 


Composite 
Test 

1=7. 

/f>3 


Reading  Summaries  (Covert  Rehearsal):  Verbal  Test 

Reading  Summaries  (Covert  Rehearsal):  Visual  Test 

Shading  on  Drawings  (Overt  Rehearsal):  Verbal  Test 

Shading  on  Drawings  (Overt  Rehearsal):  Visual  Test 
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Table  2:    Effect  of  Instructional  Treatment  on  Prior  Knowledge  Level  on 

Each  Criterion  Test 


Instructional  Treatments 


Criterion  Tests 


Treatment  1: 
Reading 
Summaries 
(Covert 
Rehearsal ) : 
Verbal  Test 


Treatment  2: 
Head  ing 
Suramarl ds 
(Coverc 
Rehearsal : 
Visual  Test 


Treatment  3: 
Shading  on 
Pictures 
(Overt 
Rehearsal ) : 
Verbal  Test 


Treatment  4: 
Shading  on 
Pictures 
(Overt 
Rehearsal) : 
Visual  Test 


Pretest 


1>2 
1>2 
2>3 


1>2 
1>3 
2>3 


1>2 
1>3 
2>3 


1>2 
1>3 
2>3 


Dra'-'ing  Test 


1>2 
1>3 


1>2 
1>3 


1>2 
1>3 


Identification 
Test 


1>2 
1>3 
2>3 


1>2 
1>3 
1>3 


1>2 


Terminology 
Test 


1>2 
1>3 


1>2 
1>3 


1>2 


Comprehension 
Test 


1>2 
1>3 
2>3 


1>2 
1>3 


1>2 
1>3 


1>2 


Composite  Test 


1>2 
1>3 


1>2 

1>3 

2>3 


1>2 


1  =  High  prior  knowledge  level 

2  =  Medium  prior  knowledge  level 

3  =  Low  prior  knowledge  level 
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ABSTRACT 

This  study  investigated  student  preferences  for  either 
teacher-written  or  computer-generated  marking  of  written 
compositions.  Ninety-seven  high  school  students  typed  assigned 
compositions  on  a  word  processor.  Three  skills  pre-selected  by 
the  teacher  were  checked  on  each  compoFltlon.  Students  we/e 
randomly  assigned  to  treatments  so  that,  on  the  first 
coTiposition,  one-half  were  marked  by  the  teacher  and  the  other 
half  by  a  computer  proofing  program.  On  the  second 
assignment,  the  teacher/computer  groups  were  switched  to  the 
opposite  type  of  marking.  On  a  third  assignment,  students 
chose  either  teacher  or  computer  marking.  Results  showed 
that,  after  experiencing  both  types  of  marking,  87%  of  the 
students  selected  teacher  marking  over  computer  marking.  The 
most  common  reasons  for  selecting  the  teacher  were  the  more 
personalized  marking,  ease  of  understanding,  additional  areas 
marked,  and  the  fact  that  the  teacher  is  the  ultimate  grader. 
Students  reported  that  they  liked  using  word  processors  (86%) 
and  that  word  processors  made  writing  easier  (93%). 
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Diagnostic  Evaluation 
of  Composition  Skills:  Student  Choice 

The  introduction  of  computers  into  language  arts  classes 
has  lagged  behind  their  earlier  use  in  such  traditionally  strong 
computer  areas  as  mathematics,  science,  and  business.  Perhaps 
because  of  the  wide  use  of  computers  for  word  processing  in 
business  classes,  the  first  uses  of  computers  in  language  arts 
were  primarily  as  word  processing  devices.  At  times  attention 
has  focused  on  the  impact  of  the  computer  itself  (Ftase,  1987) 
in  the  writing  process.  When  computers  became  more  widely 
accepted  and  less  intimidating,  educators  observed  differential 
effects  with  various  users  and  began  to  investigate  (Baer,  1988; 
Powell,  1984).  Emphasis  has  shifted  from  hardware  and  its 
effects  on  students  to  writing  and  how  computers  interact  in 
this  complex  process  (Auten,  1984;  Dees,  1985;  Wheeler,  1985). 

In  addition  to  questions  regarding  the  impact  of  both 
hardware  and  software  on  the  pedagogy  of  writing,  other 
questions  arose.  Student  attitudes  concerning  computer  use 
(Johnson,  1988;  Le,  1989;  MacArthur,  1988)  were  studied.  The 
effect  of  computers  on  student  motivation  has  been  studied  by 
Lepper  (1985)  and  by  Seymour  (1986).  Studies  by  Daly  (1978) 
and  Faigley,  Daly,  and  Witte  (1981)  dealt  with  student 
apprehension  during  the  writing  process.  However,  there  is 
evidence  that  indeed  a  place  exists  for  computers  in  teaching  in 
general  and  in  teaching  writing  specifically  (Marcus,  1987a; 
Marcus,  1987b). 

Revision  has  been  identified  as  a  desirable  writing  skill 
(Bean,  1983;  Collier,  1983;  Kurth,  1987;  Monahan,  1984; 
Murray,  1978).  The  present  study  was  the  outgrowth  of  several 
informal  projects  with  various  teachers  which  explored  ways  of 
motivating  students  in  writing  classes  to  do  more  revision. 
Previously,  students  were  reluctant  to  re-write  papers  that  had 
been  handwritten.  The  time  and  effort  of  re-writing  was  often 
beleaguered  by  new  mistakes  introduced  daring  the  re-writing 
process.  Even  when  the  students  were  provided  with  specific 
items  to  be  corrected  and  when  offered  a  higher  grade  upon 
correction  and  re-submission,  students  elected  not  to  re-write, 
instead  accepting  a  lower  grade.  Case  studies  demonstrated 
student  willingness  to  re-write  when  provided  with  opportunity 
and  facilities  such  as  a  word  processing  lab.  These  outcomes  are 
similar  to  those  described  by  Watt  (1984).  Microcomputers  have 
been  shown  to  be  effective  instruments  of  revision  (Levin,  & 
Boruta,  1984;  Levin,  Boruta,  &  Vasconcelios,  1983).  In  addition, 
computers  have  been  shown  to  play  a  positive  role  in  continuing 
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motivation  of  students  (Seymour.  Sullivan.  Story.  &  Mosley. 
1987).  In  general,  students  like  computers  (McGarvey.  1986). 

The  purpose  of  the  present  study  was  to  determine 
whether  students  preferred  to  have  their  compositions  marked 
by  their  teachers  or  by  a  computer.  After  having  a  composition 
scored  by  a  computer  and  another  by  their  teachers,  students 
wer  J  given  a  choice  between  the  computer  and  teacher  to  score 
their  third  composition.  Data  were  collected  and  analyzed  on 
their  choice,  the  reasons  for  it.  and  their  attitudes  toward  tlie 
use  of  word  processors.  Pre-project  and  post-project 
questionnaires  were  used  to  assess  student  preferences  and 
attitudes  toward  these  variables. 

Student  choice  of  evaluation  method  was  considered  to  be 
important  because  a  diagnostic  writing  lab  had  been  proposed 
for  the  English  Department  in  the  school  in  which  the  study 
was  conducted.  Students  made  their  choice  of  evaluation 
method  on  the  post-project  questionnaire.  Studies  of  continuing 
motivation  (Hemdon.  1987)  have  revealed  that  forced  choice  of 
a  method  of  instruction  or  evaluation  is  a  reliable  indicator  of 
future  student  preferences.  A  diagnostic  computer  program. 
MacProof^.  was  chosen  from  among  those  that  were  available. 
The  MacProof^v  2.0  program  was  used  to  evaluate  student 
papers  and  to  provide  written  feedback  to  the  students. 

Method 

Subjects 

Subjects  were  97  (36  male  and  61  female)  juniors  and 
seniors,  enrolled  in  four  classes  at  a  suburban  high  school  in 
Arizona.  One  of  the  classes  was  for  college-bound  seniors,  the 
other  three  were  made  up  of  average  ability  students.  The  Fifty- 
nine  of  ttic:  subjects  (61%)  reported  their  out  of  school  use  of 
computers  to  be  zero  houi^  per  week.  14  (14%)  less  than  one 
hour  per  week.  16  (17%)  between  one  and  4  hours,  and  six 
(6%)  more  than  four  hours.  The  classes  of  two  male  instructors 
were  chosen  because  of  the  instructors  prior  exposure  to 
computer  usage  for  writing  and  their  positive  orientations 
toward  students. 
Materials 

Networked  Apple  He  computers  running  AppleWorks 
word  processing  software  were  used  as  work  stations.  For  each 
assignment  three  writing  skills  were  checked.  These  skills 
corresponded  to  a  recenUy  taught  skill,  a  previously  taught  skill 
(more  than  three  weeks  prior),  and  a  universal  skill  such  as 
spelling. 
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On  exercise  one  the  three  areas  of  concentration  were 
vague  terms,  forms  of  the  ^^be"*  verb,  and  spelling.  On  exercise 
two  the  three  areas  of  concentration  were,  punctuation, 
confused  terms,  and  spelling. 

The  computer  took  each  of  the  files  that  the  students  had 
entered  and  printed  it  double  spaced.  The  files  for  the 
computer  marked  compositions  were  further  processed  with 
MacProof*.  By  selecting  each  of  the  three  skills  to  be  checked 
from  a  menu  of  twenty-one  possible  areas  and  having  the 
computer  print  out  a  list  of  errors.  Depending  on  the  type  of 
check  selected,  the  printout  would  correspond  to  one  of  the 
following:  a  list  of  misspelled  words,  or  a  questioned  item,  such 
as  a  vague  term,  highli^ted  by  boldfacing  within  a  sentence 
fragment,  coupled  with  a  short  3-5  line  explanation  of  why  the 
word  is  vague,  along  with  suggestions  on  how  to  improve  the 
sentence.  If  there  were  no  mistakes  for  a  given  skill,  there  was 
no  page  printed.  Separate  printouts  were  provided  for  each  skill 
checked  and  stapled  to  the  corresponding  composition  printout. 

Teachers  wrote  directly  on  the  student  printouts  from  the 
teacher  marked  group  using  checkmarks  and  comments 
normally  used  by  that  teacher.  In  addition  to  the  three 
designated  skills  the  teachers  often  marked  other  areas  and 
added  personalized  comments. 
Procedures 

Before  starting  the  project,  students  were  familiarized 
with  the  operation  of  the  computer  and  AppleWorks*  word 
processing  program,  including  data  entry  and  simple  editing 
procedures,  and  with  the  procedures  for  saving  and  retrieving 
files  on  a  network.  An  existing  microcomputer  lab  in  the 
Mathematics  Department  was  used  with  a  separate  ofi'-line 
process  to  evaluate  the  student  papers.  Prior  to  the  first  writing 
assignment,  the  teacher  administered  a  pre-project 
questionnaire  dealing  with  prior  computer  experience  and 
attitudes  toward  computer  usage  to  all  students.  The  mechanics 
of  saving  and  retrieving  files  from  a  network  was  also  reviewed. 

Upon  completion  of  the  pre-project  questionnaire,  the 
students  were  told  that  they  would  be  part  of  a  study  on  the  use 
of  writing  labs  and  that  the  project  would  consist  of  three 
exercises,  with  one-hsdf  of  the  papers  randomly  assigned  to 
either  teacher-marked  or  computer-marked  treatments  on 
exercise  one.  On  the  second  exercise  the  two  groups  would  be 
switched  so  that  all  students  experienced  both  methods  of 
evaluation. 
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For  each  of  the  three  exercises  the  topic  was  presented  to 
tlie  students  three  days  before  going  to  the  computer  lab.  The 
compositions  were  to  be  approximately  250-350  words  In  length 
and  composed  at  the  terminal,  although  students  were 
encouraged  to  plan  out  their  writing  ahead  of  time,  using  pre- 
writlng  activities  such  as  outlining  and  organizing  as  practiced  in 
the  usual  class  activities.  On  the  designated  day,  the  whole  class 
was  taken  to  the  lab  to  enter  their  compositions  into  the  word 
processor.  On  the  first  exercise  students  did  not  know  which 
marking  method  would  be  used  on  their  papers.  Student  files 
were  saved  to  a  central  storage  device  from  which  copies  of  the 
files  were  ported  to  a  Macintosh  computer  for  analysis  using  the 
MacProor  program.  Marked  copies  were  returned  to  students 
for  each  assignment  within  two  class  days.  The  three  exercises 
were  spaced  approximately  8-10  school  days  apart,  depending 
on  lab  availability  and  class  readiness. 

Students  selected  either  the  teacher  or  the  computer  as 
the  evaluator  of  their  draft  copy  on  the  third  written 
composition.  The  selection  took  place  after  the  students 
entered  their  composition  into  the  computer.    The  instructor 
noted  which  method  each  student  wished  to  have  applied  to  the 
material,  and  the  papers  were  marked  accordingly. 

Criterion  Measures 

The  criterion  measure  was  a  four-item  questioruiaire 
administered  to  all  students  at  the  conclusion  of  the  study.  The 
four  questionnaire  items  asked  students  (1)  Which  evaluation 
method  did  you  chose  on  cycle  three?  (2)  Word  processors 
make  wrlUng  easier  -  (Yes  or  No)?  (3)  If  a  writing  lab  were 
available  would  you  use  a  computerized  style,  grammar,  and 
spelling  checker  for  assignments?  and,  (4)  What  made  you 
choose  either  the  computer  or  teacher  evaluation  for  the  third 
cycle? 

Post-Project  Questionnaire  responses  to  "What  made  you 
choose  e^  'her  the  computer  or  teacher  evaluation"  were 
classified  by  the  experimenter  as  pro  or  con  teacher  and  pro  or 
con  computer. 

A  five-item  Pre-Project  Questionnaire  was  also 
administered  to  all  students.  The  Pre-Project  Questionnaire 
dealt  with  student  attitudes  toward  computers  and  student  use 
of  and  experience  with  computers  and  word  processing,  as  well 
as  with  student  attitudes  toward  teacher  grading  of  papers  and  a 
student  self-ranking  of  writing  ability. 
Dataanalvsis 
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All  questionnaire  responses  were  analyzed  using  Chi- 
square. 

Results 

Eighty-four  of  the  97  students  (87%)  chose  to  have  the 
teacher  mark  their  third  composition  and  13  students  (13%) 
chose  to  have  the  computer  mark  it.  Chi-square  analjf.is 
revealed  that  this  preference  for  teacher  over  computer 
evaluation  was  statistically  significant,     (1.  N=97)  =  51.97, 
p<.001. 

Student  responses  to  the  question  "What  made  you  choose 
either  the  computer  or  teacher  evaluation  for  the  third  cycle 
[exercise]?"  were  classified  into  common  categories  for  ease  of 
understanding  and  reporting. 

Students  wrote  a  total  of  70  pro-teacher  statements  and 
18  pro-computer  statements.  Students  also  wrote  16  negative 
statements  about  the  computer,  but  none  about  the  teachers. 
The  frequency  of  each  type  of  response  is  shown  in  Table  1 .  It 
can  be  seen  from  the  table  that  the  most  frequent  pro-teacher 


PLEASE  INSERT  TABLE  1  ABOUT  HERE 


responses  were  personal  help  (24  responses)  and  ease  of 
understa./ding  (23  responses).  Other  frequent  pro-teacher 
responses  were  that  the  teacher  checked  additional  areas  (15 
responses)  and  that  the  teacher  is  the  ultimate  grader  (12 
responses).  The  most  frequent  pro-computer  response  (7 
students  or  7%)  was  that  it  catches  more  mistakes  than  the 
teacher,  and  the  most  common  con-computer  response  was  that 
it  catches  too  many  mistakes  (8  responses).  There  were  no  con- 
teacher  responses. 

Responses  to  several  other  questions  on  the  Pre-Project 
Questionnaire  and  Post-Project  Questionnaire  are  summarized  in 
lable  2.  On  the  Pre-Project  Questionnaire  83  of  the  97  students 


PLEASE  INSERT  TABLE  2  ABOUT  HERE 


(86%)  reported  that  the  liked  using  word  processors  and  14 
(14%)  reported  that  they  did  not.  Seventy-one  students  (73%) 
responded  that  they  thought  typed  papers  received  higher 
grades  than  handwritten  ones,  whereas  26  (27%)  indicated  they 
did  not  think  so.  The  difference  between  teachers  on  this  item 
was  not  statistically  significant. 
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Table  1 

Student  Reasons  for  Choice  of  Teacher  or  Computer 


Pro  Teacher  Number 

Personal  help  24 

Easier  to  understand  23 

Additional  areas  checked  (style  etc.)  15 

Ultimate  grader  of  end  product  12 

Specific,  in  place  comments  7 

Alive/human  grader  4 

Pro  Computer 

Thorough  -  catches  more  mistakes  than  teacher  7 

Lists  mistakes  5 

Efficient  5 

Easy  4 

Con  Computer 

Too  many  mistakes  (non-mistakes  listed)  8 

Too  confusing  5 

Con  Teacher 
None 
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Table  2 

Student  Attitudes 


Question  Yes  No 


Pre -3  Like  using  word-processors 

83 

14 

(86%) 

(14%) 

Pre-4  Higher  grades  -  typed  papers 

71 

26 

(73%) 

(27%) 

Post-2  Word  processors  make  writing  easier 

90 

7 

(93%) 

(7%) 

Post-3  Use  lab  (w/more  features)  if  available 

76 

21 

(78%) 

(22%) 

Male 

25 

11 

(69%) 

(31%) 

Female 

51 

10 

(84%) 

(16%) 
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On  the  Post-Project  Questionnaire,  90  students  (93%) 
responded  that  they  thought  word  processors  make  writing 
ea3ler,  whereas  only  seven  students  thought  they  did  not. 
Seventy-six  students  (78%)  reported  that  they  would  use  a 
writing  lab  if  or.^  were  available,  and  21  students  (22%) 
reported  that  they  would  not.  A  significantly  higher  number  of 
girls  (84%)  than  boys  (69%)  reported  that  they  would  use  the 

writing  lab,     (1.  N=97)  =  16.7,  p<.005. 
Discussion 

The  primary  result  of  this  study  strongly  favored  teacher 
marking  of  student  compositions  over  computer  marking. 
Students  liked  the  more  personal  nature  of  the  feedback  from 
the  teachers.  They  also  preferred  teacher  marking  because  it 
was  easier  to  understand,  perhaps  because  the  nature  of  the 
process  enables  the  teacher  to  provide  more  specific  comments 
at  the  relevant  points  in  the  manuscript  whereas  the  computer 
feedback  is  more  general  and  on  a  separate  page.  Many  students 
also  liked  the  fact  that  the  teachers  checked  additional  areas 
than  those  emphasized  for  the  particular  composition  or 
preferred  teacher  marking  because  the  teacher  was  their 
"writing  audience"  -  -  the  ultimate  grader.  Several  students 
liked  the  computer  for  its  thoroughness  in  catching  mistakes; 
whereas  several  others  thought  it  was  too  thorough,  catching  too 
many  mistakes  or  some  non-mistakes. 

Although  students  preferred  teacher  marking  over 
computer  marking,  data  from  the  study  indicate  very  positive 
student  attitudes  toward  the  use  of  computers  for  preparing 
their  compositions.  Ninety-three  percent  of  the  students 
thought  that  word  processors  made  writing  easier. 

Seventy-three  percent  of  the  students  indicated  that  they 
thought  typed  papers  would  receive  higher  grades  over 
handwritten  papers.  During  the  study  all  of  the  student  papers 
were  mechanically  reproduced  as  a  result  of  using  word 
processors,  and  the  instructors  corajnented  that  mechanical 
printing  made  the  papers  easier  and  faster  to  read. 

Overall,  78%  of  the  students  responded  that  they  would 
use  a  writing  lab  if  one  were  available.  Since  the  completion  of 
this  project,  a  twenty-one  station  Macintosh  equipped  writing 
lab  has  been  installed  at  the  school.  Personal  observation  and 
network  server  logs  indicate  that  the  lab  is  busy  before,  during, 
and  after  school.  There  was  a  difference  by  sex  on  using  a 
writing  lab  with  84%  of  the  females  indicating  that  they  would 
use  a  lab  while  69%  of  the  males  favoring  use  of  a  lab. 
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Personal  help  was  the  reason  given  most  frequently  by 
students  for  choosing  teacher  marking.  Examination  of  teacher- 
marked  papers  revealed  that  both  teachers  often  used  students* 
first  names  when  writing  comments  on  the  papers.  The 
teachers  often  included  references  to  other  personal  matters  or 
events,  and  may  thereby  have  increased  students*  feelings  of 
personalization.  Of  course,  just  the  more  personal  nature  of 
marking  by  a  human  being  rather  than  a  computer  may  also 
contribute  to  those  feelings. 

Personalization  of  instruction  can  have  an  important  effect 
on  student  learning  (Anand,  &  Ross.  1987;  Ross,  &  Anand, 
1987).  Other  researchers  (L6pez,  1989)  have  generated 
personalized  instructional  programs  by  computer  that  produced 
better  student  achievement  and/or  attitudes  than  non- 
personalized  programs.  Personalization  in  these  studies  was 
accomplished  by  having  each  student  complete  an  inventory  of 
interests,  including  the  names  of  friends,  which  were  then 
incorporated  into  individualized  learning  materials  for  each 
student.  Although  these  studies  were  conducted  in  the  areas  of 
math  and  logic,  similar  effects  may  have  occurred  with  the  two 
teachers  in  this  study  when  they  used  the  students'  first  names 
and  personalized  events  when  adding  comments  to  the  marked 
papers. 

Personalization  of  feedback  by  computer  on  complex  tasks 
may  be  more  difficult  to  accomplish  efficiently  than 
personalization  of  initial  instruction.  Merely  inserting  the 
student's  name,  form  letter  style,  into  the  computer  analysis  of 
the  composition  would  probably  not  have  the  same  effect  as  the 
teacher's  personalized  comments  written  directly  on  the 
students'  papers.  Despite  advances  in  computer  systems,  it 
would  seem  that  when  dealing  with  student  writing  the  teacher 
can  react  more  appropriately  to  student  needs,  pinpointing 
problems  and  suggesting  specific  remedies,  while  maintaining 
rapport  with  the  student. 

The  teachers  were  free  to  provide  other  types  of  feedback 
to  the  students  on  topics  such  as  overall  style.  The  students 
noted  tills  in  their  questionnaire  responses.  In  post-project 
interviews  students  commented  that  they  were  writing  to  the 
teacher's  requirements.  The  teacher  was  the  grader  of  the 
paper,  and  that  if  the  teacher  pointed  out  areas  that  needed 
additional  work,  this  was  all  that  needed  fbdng.    Papers  would 
receive  a  hi^er  grade  if  the  student  fixed  what  the  teacher 
marked.  These  facts  should  diminish  any  fears  that  when 
students  use  computers  with  proofing  aids  in  language  arts 
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classes     .  .  there  won't  be  anything  left  to  correct"  (C.  Crowe, 
personal  timmunication,  1985). 

Students  also  liked  the  way  that  the  teacher  comments 
were  placed  directly  on  their  papers,  very  near  the  problem. 
Students  commented  that  they  liked  the  giader  to  be  alive 
and/or  human.  Other  students  commented  that  they  thought 
teachers  stopped  checking  after  four  or  five  mistakes.  Some 
noted  that  the  computer  was  exhaustive  in  finding  all  mistakes. 

Several  style-checker  programs  are  currently  available. 
Most  are  specific  to  a  particular  type  of  computer.  MacProof*v 
2.0,  the  checker  program  used  In  this  study,  was  compatible 
within  the  family  of  classroom  computers  available  at  the  school. 

The  two  teachers  selected  to  participate  in  the  study  were 
young  (24-26  years)  male  teachers  recognized  for  their  ability  to 
work  well  with  students.  The  popularity  of  these  two  teachers, 
familiarity  with  students,  and  the  use  of  personalized  remarks  on 
papers  may  have  influenced  the  selection  made  by  students. 

The  teachers  liked  having  the  computer  provide 
rudimentary  analyses  of  the  student  papers  in  basic  skills  such  as 
spelling,  punctuation,  vague  terms  and  other  areas,  which  freed 
up  the  teacher  to  work  with  students  on  "Idea/creative"  areas  of 
the  assignments. 

In  her  chapter  on  Writing,  Ellen  Gagne  (1985)  points  out 
that  unskilled  writers  spend  much  of  their  time  trying  to  avoid 
mechanical  mistakes.  Proofing  programs  allow  writers  to 
concentrate  more  on  commimlcating  well,  leaving  mechanics  to 
be  checked  prior  to  the  final  draft.  Wlien  the  students  are 
working  at  the  computer,  entering  the  composition,  the 
teachers  were  able  to  circulate  amongst  the  students,  providing 
individual  attention.  Butler  (1985)  would  see  this  as  highly 
supportive  of  many  of  the  features  listed  by  Hairston  (1982)  as 
characteristics  of  the  new  composition  paradigm. 

One  observation  from  the  study  was  that  teacher-marking 
rad  computer-checking  In  combination  may  be  the  most 
effective  way  to  mark  compositions.  Based  on  the  student 
selection  of  the  teacher  as  grader  and  the  teacher  acceptance  of 
the  time  saving/error  catching  capabilities  of  the  computer 
program  an  effective  combination  would  have  both  the  teacher 
and  the  computer  available  to  students.  The  computer  serving 
the  student  as  a  tool  in  the  writing,  editing,  and  revising  cycle, 
while  the  teacher  provides  overall  instruction  and  personsdlzed 
marking  of  papers.  Version  3.2  of  the  MacProof*  program 
operates  as  an  interactive  desk  accessory.  Students  would  be 
encouraged  to  pre-check  their  own  papers  with  the  help  of  the 
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computer  to  eliminate  grammatical  mistakes,  allowing  teachers 
to  spend  more  time  helping  the  students  write  better. 

Additional  research  is  needed  to  test  the  effectiveness  of 
combining  teacher  personalization  of  feedback  with  the  analytic 
capabilities  of  existing  computer  software.  Combinations  of 
direct  student  access  of  composition  mechanics  coupled  with 
degrees  of  teacher  involvement  during  the  revision  process 
could  yield  guidelines  on  how  students  can  best  be  served  by  the 
availability  of  writing  laboratories  and  computers. 
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Purpose  of  This  Study 

The  EducaLional  Resources  Information  Center  (ERIC)  attempts  ^,o  generate 
syntheses  of  the  literature  of  various  academic  disciplines  and  other  areas 
that  contribute  to  the  field  of  Education.     The  purpose  of  this  publication 
is  to  provide  some  indication  as  to  where  the  field  of  educational 
technology  is  going.     The  identification  of  emerging  trends  and  issues  is 
first  step  in  this  endeavor. 

The  ERIC  Clearinghouse  on  Information  Resources  specializes  in  the  areas  of 
educational  technology  and  library/ information  science.     One  objective  of 
this  study  is  to  provide  an  indication  of  the  emerging  trends  and  issues  in 
the  field  of  educational  technology.    A  parallel  study  of  trends  and  issu  ; 
in  library  and  information  science  was  conducted  at  the  same  time  using  ^  3 
same  methodology.     Findings  from  that  study  are  reported  in  IR  No,  81, 
Trends  and  Issues  in  Library  and  Information  Science  1988,  which  is 
available  from  the  ERIC  Clearinghouse  on  Information  Resources,     Because  the 
sample  of  the  literature  that  was  included  in  this  study  was  drawn  from  a 
single-year  period  (October  1987-September  1988),  the  ability  to  verify 
trends  is  somewhat  limited.     Another  objective  of  this  particular  study, 
then,  is  to  develop  a  database  that  will  serve  as  a  reference  point  for 
trends  in  the  area  of  education  technology  to  facilitate  their 
identification  and  analysis  in  the  future,     A  third  objective  is  to  develop 
a  methodology  for  the  identification  of  emerging  trends  that  would  allow  a 
large  number  of  documents  to  be  reviewed  in  a  reasonably  short  period  of 
time  without  relying  too  heavily  on  purely  quantitative  forms  of  analysis. 

Content  Analysis:    Some  Background 

Content  analysis  was  used  to  determine  the  emerging  trends  and  issues 
relevant  to  educational  technology,    A  trend  is  considered  to  be  a 
cumulative  indicator  of  activities  or  products  that  shows  direction.  An 
issue  is  considered  to  be  a  problem  or  a  question  for  which  there  are 
multiple  points  of  view.     It  is  important  to  note  that  a  trend  may  be 
considered  by  some  to  be  an  issue.     As  a  problem  or  question  develops  within 
an  academic  field,  it  may  be  considered  an  issue.     The  distinction  between 
these  two  concepts  is  not  as  clear  as  one  might  like  it  to  be. 

Content  analysis  is  intended  to  be  a  method  for  the  objective  and  systematic 
collection  of  pre-specif ied  data  for  the  purpose  of  identifying  the  special 
characteristics  of  those  data  (Carney,  1972),     It  is  a  broad  concept  that 
can  be  used  in  any  attempt  to  practice  research  or  science.  Whenever 
symbolic  action  or  communication  is  the  subject  of  investigation,  the 
analysis  of  content  is  involved  (Janowitz,  1976), 

Not  all  content  v-'nalysis  is  the  same:  there  are  both  quantitative  and 
qualitative  strains  of  concent  analysis.    A  common  form  of  the  qualitative 
type  of  content  analysis  is  conceptual  analysis,  i,e.,  the  investigation  of 
the  use  and  meaning  of  particular  words,    A  common  type  of  quantitative 
content  analysis  is  the  measurement  of  the  length  of  articles  that  deal  with 
O  „       a  predetermined  topic,     A  tabulation  of  the  results  of  either  of  these  forms 
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of  coiiLent  analysis  is  required  in  order  to  make  any  inferences  about 
tfonds.    Examples  of  the  use  of  content  analysis  as  a  methodology  have 
appeared  in  both  the  popular  and  the  academic  press.     John  Naisbctt's 
Megatrends  (1982)  identified  10  trends  that  he  thinks  will  become 
influential  in  our  lives. 

Morris  Janowitz  (1976)  outlines  the  application  of  content  analysis  to 
determine  socio-political  trends  from  a  sampling  of  our  nation^s  newspapers. 

Reviews  of  the  advantageb,  limitations,  and  features  of  content  analysis  can 
be  found  in  Janowitz  (1976),  Carney  (1972),  and  Hosti  (1969). 

Content  Analysis  in  This  Study 

The  professional  literature  of  a  field  indicates  the  concerns,  inquiries, 
and  research  that  are  important  to  that  field.    Such  is  the  case  with  the 
literacure  of  educational  technology.     It  was  decided  that  a  review  of  the 
professional  literature  of  educational  technology  would  reveal  the  ideas 
that  were  of  importance  to  the  field.     Through  this  piocess  it  was  hoped 
tiiat  indications  could  be  derived  about  the  direction  (emerging  trends  and 
Issues)  of  the  field.     The  basic  idea  behind  this  study  was  that  if  one 
could  classify  and  tabulate  pre-selected  writings  based  on  interpretations 
of  the  authors'  purpose  it  would  be  possible  to  indicate  current  directions 
in  the  field. 

In  order  to  achievre  this  end,  it  was  determined  that  a  substantial  number  of 
source  items  would  have  to  be  reviewed.    Because  this  study  was  primarily 
concerned  with  the  number  of  times  that  a  topic  was  discussed  (as  opposed  to 
the  length  of  the  discussion  or  the  particular  use  of  the  concepts  involved 
in  the  discussion),  and  given  the  resource  constraints  (limited  time  and 
manpower),  it  was  decided  that  a  content  analysis  would  be  the  appropriate 
data  collection  technique,  but  that  a  "hybrid**  form  of  content  analysis 
would  have  to  be  developed  for  the  purpose  of  this  study. 

It  was  imperative  that  the  data  collection  teams  be  able  to  review  and 
classify  sources  efficiently  and  effectively,  while  avoiding  the  pitfall  of 
playing  "fast  and  loose'*  with  the  data.    The  development  of  this  "hybrid" 
form  of  content  analysis  can  be  outlined  as  follows: 

•  The  conceptualization  of  the  recording  units  and  categories. 

•  The  determination  of  the  rources  to  be  reviewed. 

•  The  specification  of  the  data  collection  procedures. 
^       The  analysis  of  the  data. 

It  is  reasonable  to  state  that  the  vast  majority  of  content  analyses  that 
are  concerned  with  determining  trends  must  somehow  address  these  steps. 
However,  these  particular  tasks  do  not  always  occur  in  the  order  in  which 
they  were  performed  in  this  study.    A  reordering  of  tasks  can  be  the 
difference  between  inductive  and  deductive  studies.     In  this  study  we  use  a 


iloduct  ive  mo.tliodologv.     Because  oi  the  nature  of  Lho  contenL  I'laL  is 
specific  lo  each  field  \n  Lronds  rebeci?-ch,  there  is  p    one  right  way  to  do 
cop.tonL  I  rondb  analysis.     Henco,  by  necessity,   it  is  brid  methodology. 

It    ii>  imporlanl  to  acknouiedgo  tliat  each  type  of  coiH.       analysis  has  its 
ow!!  advantages. 

Concepr.ua  1  Categories  into  Operational  De^^  'tions 

The  ger;jral  content  categories  that  were  used  in  this  study  were  based  on 
tihe  concept  of  the  functions  performed  by  media  personnel  that  were 
discussed  by  Chis'noim  and  Ely  in  1976.     This  conceptual  bcheme  reflects  the 
definition  of  educational  technology  put  forth  by  the  Association  for 
Educational  Communications  and  Technology  (1977).     These  general  areas  were 
thought  to  be  indicative  of  the  field  of  educational  technology:  Personnel, 
Management,  The  Field,   Instructional  Processes,  Information  Seuvices, 
Tecl^nical  Developments,  and  F  search  and  Theory.    The  content  categories? 
were  determined  before  the  study  began  rather  than  during  the  review  of  the 
data  sources.     In  tliis  sense  the  categories  were  "imposed"  upon  the  study 
ratlier  than  generated  in  a  more  inductive  fashion.     This  conscious  decision 
was  based  on  the  need  for  efficiency  in  che  data  collection  process.  After 
the  general  content  categories  had  been  identified,  subcategories  were 
specified.     Some  were  added  later  during  the  data  collection  process.  The 
specification  of  these  components  was  based  on  a  thorough  knowledge  of  the 
literature  and  extensive  experience  in  professional  practice. 

An  example  of  one  of  the  conceptual  categories  is  the  function  of 
"management."    The  broad  concept  of  management  was  operationalized  by 
specifying  the  following  tasks:     planning,  budgeting,  diffusion  and 
implementation,   logistics,  operations,  and  facilities. 

Two  instruments  had  to  be  created  for  this  study.     The  primary  instrument 
allowed  reviewers  to  record  both  the  source  of  the  data  and  the  category 
into  which  the  source  had  been  classified.     The  second  instrument  served  as 
a  tally  and  comment  sheet  for  each  item  collected  and  classified  on  the 
primary  instrument. 

Content  Source  Units 

Journal  articles,  dissertation  abstracts,  ERIC  documents,  and  professional 
conference  programs  were  chosen  as  the  content  units  because  they  provide  a 
current  record  of  issues  and  topics  that  leaders  in  the  field  of  educational 
technology  have  acknowledged  as  important. 

The  analysis  cf  the  content  in  professional  journal  articles  and 
dissertation  abstracts  is  not  new  to  research  in  the  area  of  educational 
technology. 

Torkelson  (1978)  reviewed  15  years  of  the  Audiovisual  Conununi cat  ions  Review 
(AVCR)  as  part  of  an  investigation  to  determine  which  issues  had  had  the 
strongest  impact  on  the  development  of  educational  technology  as  a 
professional  field.     His  article  covers  his  AVCR  findings  as  well  as  trends 
and  areas  identified  for  further  study. 
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Mayo  (1976)  analyzed  AVCR  journals  published  in  a  20-ycar  period  between 
1953  and  1972  for  a  doctoral  dissertation.     The  study  showed  several  treids 
within  the  context  of  the  journals.     These  included  an  increase  in  the 
sophistication  of  research  design,  the  methodology,     nd  the  statistical 
analysis . 

Lard  (1979)  reviewed  two  journals.  Audiovisual  Communications  Review  (AVCR) 
and  Audiovisual  Instruction  (AVi),  the  official  journals  of  the  AECT.  The 
scope  of  her  study  also  included  the  abstracts  of  dissertations  from  five 
universities  with  major  doctoral  programs  in  the  field  of  educational 
technology.     T'nis  study  revealed  that  there  were  three  distinct  paradigms 
operating  in  the  field  of  educational  technology  during  the  21'year  period 
covered  by  the  study:     the  media  movement,  systems  theory,  and  behavicral 
technology. 

The  journals  selected  for  analysis  in  this  study  were  those  that  had  been 
identified  as  five  of  the  "most  influential"  professional  journals  by  Moore 
in  1981  and  Moore  and  Braden  in  1987:    The  Journal  of  Instructional 
Development,  The  Educational  Communications  and  Technology  Journal  (formerly 
AVCR),  Educational  Technolot>y,  TechTrends,  -.nd  The  British  Journal  of 
Educational  Technology. 

Tlie  dissertations  that  were  included  in  this  study  were  produced  at  the 
universities  that  were  identified  by  Moore  in  1981  and  Moore  and  Braden  in 
1988  as  being  the  "most  prestigious  institutions"  in  the  field  of 
instructional  technology:    Arizona  State  University,  Florida  State 
University,  Indiana  University,  Syracuse  University,  and  the  University  of 
Southern  California. 

Conference  programs  and  ERIC  documents  were  added  to  this  particular  study 
in  order  to  broaden  the  scope  of  the  content  to  be  analyzed.     It  appeared 
that  conference  programs  would  reveal  the  latest  developments  in  the  field 
of  educational  technology  because  conference  presentations  usually  discuss 
the  most  recent  findings  of  current  research  and  development  efforts.  Three 
professional  conference  programs,  the  Association  for  Educational 
Communications  and  Technology  (AECT),  the  National  Society  for  Performance 
and  Instruction  (NSPl),  and  the  Educational  Technology  International 
Conference  (ETIC)  were  included  in  the  database  for  this  study.     The  AECT 
and  NSPI  programs  were  included  here  because  these  two  organizations  are 
considered  to  be  the  two  most  prestigious  organizations  in  the  area  of 
educational  technology  in  the  United  States.     The  AECT  conference  is 
considered  to  be  the  one  for  those  interested  in  the  academic  aspects  of  the 
field  of  educational  technology,  while  the  NSPI  conference  is  considered  to 
be  the  one  that  is  attended  by  practitioners  of  educational  technology.  Tl;e 
ETIC  conference  program  was  included  in  this  study  to  ensure  a  more  global 
outlook  on  the  emerging  trends  and  issues  in  the  field. 

ERIC  documents  were  included  in  the  scope  of  this  inquiry  since  the 
materials  entered  in  the  ERIC  database  represent  a  cross-section  of  the 
contemporary  literature  of  educational  technology.     Following  the  basic 
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promise  of  Webb  (1966),  the  research  Ltaam  believed  thai  a  puiitiple 
operational  approach  should  be  emulated. 

The  research  team  felt  that  it  could  liave  an  increased  level  of  confidence 
in  its  analysis  if  it  increased  t\\e  scope  and  breadth  of  the  data  included 
in  Lne  study. 

Data  Collection  Procedures 

The  data  classification  and  tabulation  instrumencs  were  tested  for  their 
functionality.     This  was  done  by  asking  prospective  data  collectors  to  use 
the  instrument  to  review  the  same  tliree  articles  from  one  professional 
journal.     Particular  attention  was  paid  to  the  conceptual  clarity  of  the 
content  areas,  and  efforts  were  made  to  identify  ambiguities  and  confusing 
elements  within  the  instruments  that  might  mislead  the  data  collectors  into 
generating  erroneous  classifications.     The  graphic  design  of  the  instruments 
was  checked  to  insure  the  efficiency  and  the  effectiveness  of  the  data 
recording  process. 

A  training  session  was  held  for  the  data  collectors,  who  were  all  graduate 
students  at  Syracuse  University.     This  session  was  designed  to  teach  tliem 
how  to: 

^        Identify  the  purpose  of  an  article  by  reading  the  introduction, 
abstract  and  concluding  statement. 

9        use  tlie  data  source  and  classification  instrument. 

Use  the  tabulation  and  comment  instrument. 

•        Locate  the  data  sources. 

The  ability  of  the  data  collectors  to  meet  these  four  objectives  were 
demonstrated  by  a  test  for  inter-rater  reliability. 

After  a  period  of  one  week  the  data  collectors  met  and  compared  the  results 
of  their  interpretations  of  the  articles  from  three  different  professional 
journals.     The  Pearson  Correlation  Coefficient  (r)  that  had  been  calculated 
for  the  pairs  of  data  generated  by  the  collectors  slightly  exceeded  r=0.8. 
To  insure  further  reliability,  it  was  decided  that  each  data  collector  would 
designate  two  possible  categories  for  each  of  the  journal  articles,  a 
primary  choice  and  a  secondary  choice.     The  employment  of  this  technique 
virtually  eliminated  cases  of  non-agreement  among  the  data  collectors. 
Inter-rater  reliability  was  virtually  assured  as  data  collectors  discussed 
those  cases  where  they  had  disagreed  until  an  agreement  could  be  reached. 

At  the  conclusion  of  the  data  source  recording  and  classification  phase  of 
the  study,  the  data  collectors  tabulated  the  results.    They  reported  the 
results  of  all  of  the  dissertations  on  one  tabulation  instrument  for 
analysis.     Each  of  the  five  professional  journals  and  each  of  the  three 
conference  programs  that  were  reviewed  was  reported  separately,  as  were 
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monthly  entries  in  the  ERIC  index,  Resources  in  Education  (RIE).  This  style 
of  reporting  allowed  for  cross-source  analysis. 

Majur  reports  and  position  papers  published  within  the  time  period  of  the 
study  and  personal  observations  from  professional  participation  in  national 
and  international  events  were  also  used  to  provide  further  input  and 
clarification. 

Limitations  to  the  Study 

There  are  a  number  of  limitations  to  this  study.     First,  as  a  form  of 
content  analysis,   the  study  is  a  "hybrid."  It  does  not  enjoy  the  advantages 
of  conceptual  analysis,  i.e.,  the  inquiry  into  the  use  and  meaning  of 
particular  terms  that  have  a  bearing  on  the  field  of  educational  technology. 
Second,   it  has  no  quantitative  base  other  than  that  of  tabulation.  The 
methodology  did  not  reveal  the  depth  of  the  particular  content  that  was  used 
or  analyzed  in  the  sources.     Third,  the  decision  to  use  sources  that  have  no 
precedent  in  content  analysis  (e.g.,  conference  programs)  raises  questions 
about  the  data  sources  that  were  used  in  the  study.     Fourth,  there  is  the 
possibility  that  certain  items  that  were  reviewed  as  source  data  were 
"specialty  items."    Special  issues  of  a  particular  jovrnal  or  a  conference 
dedicated  to  a  particular  theme  could  "skew"  the  results  of  a  study.  Fifth, 
it  is  difficult  to  make  statements  about  trends  based  on  the  data  gathered 
in  a  single  year  since  tliere  is  no  earlier  referent.     Finally,  the  attempt 
to  emulate  the  multiple  operational  model  of  analysis  was  not  a  true  one. 
Since  multiple  operationalism  is  an  attempt  to  bring  several  different 
nethodologies  to  oear  on  a  particular  question  or  concern,  this  study  could 
be  considered  as  a  cross-methodological,  meta-analysis.     Although  attention 
was  given  to  increasing  and  varying  the  amount  and  types  of  data  sources 
used  to  analyze  trends  and  issues  in  the  field  of  educational  technology, 
the  study  used  a  single  methodology. 

Recommendations  for  the  Future 

It  is  more  than  an  exercise  in  humility  to  acknowledge  the  limitations  of  a 
particular  study:   it  is  intellectual  honesty.     Three  goals  were  set  forth  at 
the  beginning  of  this  study.     The  first  was  to  provide  an  indication  of  the 
emerging  trends  and  issues  in  the  field  of  educational  technology.  The 
difficulty  of  recognizing  trends  based  on  data  gathered  from  a  relatively 
short  time  span  has  already  been  mentioned.     But  certainly  the  number  and 
variety  of  sources  utilized  make  the  analysis  provided  here  more  than  an 
"educated  guess."  The  second  goal  of  this  study  was  to  develop  a  data  source 
that  will  serve  as  a  baseline  so  that  trends  in  the  field  of  educational 
technology  can  be  more  easily  recognized  in  the  future.     It  is  much  easier 
to  have  confidence  in  having  met  this  second  goal.    The  data  that  have  been 
gathered  during  the  course  of  this  study  will  presumably  be  used  by  future 
analysts,  as  they  provide  a  tabulation  of  the  professional  discussions  about 
what  comprised  the  field  of  educational  technology  in  1987-88,     The  third 
goal  of  this  study  was  to  develop  a  methodology  that  will  allow  an  expedient 
review  of  the  professional  literature  of  the  field  of  educational 
technology.     Here,  too,  it  is  easy  to  feel  confident  about  having  met  the 
goal;  however,  there  is  still  much  to  be  done  in  this  regard.  The 
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instruments  require  some  refinement.     Data  sources  must  be  reconsidered. 
Integration  of  other  types  of  trond  and  issue  analysis  inco  this  methodology 
should  be  addressed.     Webb,  Campbell,  Schwart..,  and  Sechrest  recommended  the 
technique  of  muUiple  operat  ionalisin,  asserting  that  "once  an  idea  has  been  ^ 
confirmed  by  two  or  more  in  dependent  measuring  processes  the  uncertainty  of 
itb  interpretation  will  have  been  greatly  reduced"  (1966,  p. 3).  Resour'::e 
and  time  constraints  precluded  the  use  of  other  data-gathering  techniques 
and  contribute  to  liniitations  of  this  study.     The  decision  to  broaden  the 
i>cope  and  number  of  content  sources  sampled,  however,  increases  the  level  of 
confidence  that  can  be  placed  in  the  data.     These  are  questions  that  must  be 
faced  at  the  beginning  of  any  such  trend  analysis.     Surely  more  such 
questions  will  emerge..    Upon  the  completion  of  this  phase  of  this  infinite 
enterprise  one  can  only  think  that  long  journeys  start  with  small  steps. 

Major  Trends  Identified  by  the  Study 

Design,  development,  and  evaluation  of  instructional  materials  and 
procedures  is  a  primary  concern  among  practitioners  in  the  field  of 
educational  techno  logy . 

A  large  portion  of  the  educational  technology  literature  is  about  the 
design,  development,  and  evaluation  of  instructional  materials.     Issues  in 
design  include  the  application  of  cognitive  psychology,  such  as  in  helping 
learners  to  conceptualize  unfamiliar  content;  semiotics  and  the  effects  of 
message  configuration  characteristics,  such  as  text  design,  text  density, 
visual  design,  and  use  of  s>Tnbol  systems;  and  the  effects  of  media  use  on 
motivation,  including  learner  interest,  achievement,  and  attitude 
development.     Development  includes  such  activities  as  needs  assessment, 
course  development,  and  product  development.     Finally,  evaluation  is 
concerned  with  measures  and  procedures  for  determining  program 
effectiveness.     Related  to  this  is  a  call  for  better  procedures  for 
evaluating  computer-assisted  instruction  software  and  software  evaluation 
databases  that  are  accessible  to  individuals. 

Professional  education  for  teachers  in  the  use  of  educational  technology 
principles  and  practices  is  seen  as  a  basic  need  for  present  and  future 
professional  service. 

Literature  in  this  area  is  directed  both  at  professional  specialists  within 
the  field  of  educational  technology  and  at  individuals  who  teach. 
Currently,  the  emphasis  appears  to  be  on  the  teacher/ instructor .     The  basic 
question  is,  "What  competencies  do  teachers/ instructors  need  to  use 
technology  effectively  with  their  learners?"    The  assumption  is  that  all 
classroom  presenters  should  be  using  media  and  technology  but  are  not,  or 
that  they  are  using  it  in  less  than  optimal  ways.^  Much  of  the  literature 
discusses  the  use  of  computers  and  microcomputers  by  classroom  teachers,  and 
it  emphasizes  the  need  for  teacher  training  in  the  area  of  information 
technology  rather  than  educational  technology  or  library  instruction.  This 
subtle  difference  is  indicative  of  the  gradual  blending  of  educational  media 
and  technology  with  library  and  information,  science  in  the  elementary  and 
secondary  scliools. 
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Distance  education  is  becoming  a  significant  instructional  delivery  system 
that  uses  technological  means  to  reach  its  goals. 

Interest  in  distance  education  is  stimulated  in  part  by  concerns  over  equity 
of  access  in  the  face  of  shortages  of  qualified  elementary  and  secondary 
school  teachers.     Distance  learning  protocols  liave  developed  in  direct 
response  to  real  teaciung/ learning  problems.    Distance  education  offers 
practicable  solutions  to  shortages  of  resources  and  teaching  personnel.  To 
prepare  material  f:>^  delivery  requires  a  systematic  approach  to 
instructional  design  and  a  concern  for  the  individual  student  rather  than 
for  group  teaching.     Much  of  the  literature  about  distance  education  refers 
to  the  use  of  various  telecommunications  systems  to  provide  optimum 
participation  by  the  learners  (National  Governors*  Association,  1988). 

The  computer  is  the  dominant  medium  in  the  field  of  educational  technology. 

Statistics  show  tremendous  growth  in  school  use  of  computers  in  rt-^nt  years 
(Quality  Education  Data,   1988).     It  is  no  longer  just  the  computer- 
established  secondary  schools  that  lead  the  field;  now  more  than  half  of 
U.S.  elementary  schools  have  enough  computers  to  provide  at  least  one  for 
every  two  classrooms.     Elementary  schools  use  their  micros  primarily  to 
supplement  lessons  with  basic  skills  exercises  and  opportunities  for  drill 
and  practice.     Ii  secondary  schools,  the  micros  are  used  primarily  for 
teaching  formal  computer  literacy  (Talmis,  1988).     Teachers  express  interest 
in  having  publishers  develop  software  that  teaches  problem-solving  skills 
and  higher  order  thinking  skills.     While  the  literature  reflects  the  growing 
enthusiasm  for  school  computer  use,   it  also  shows  continuing  criticism  of 
software  quality. 

After  computers,  telecommunications  and  video  are  emerging  as  major  media 
delivery  systems. 

The  apparent  preoccupation  of  educators  with  computers  often  overshadows  the 
increasing  interest  in  telecommunications  and  video,    \fliile  schools  continue 
to  use  the  traditional  audiovisual  equipment  such  as  films,  filmstrips, 
slides,  audiotape  recordings,  and  overhead  transparencies  in  a  more-or-less 
routine  fashion,  new  development  seems  to  be  with  video  in  the  classroom  for 
large  group  instruction,  and  telecommunications  for  individuals  and  small 
groups  within  the  school  and  in  distance  education  programs*    The  Quality 
Education  Data  study  (1988)  notes  that  over  90%  of  the  schools  in  the  U.S. 
are  using  videocassette  recorders,  while  in  1983  only  30%  of  the  schools 
used  VCRs. 

The  role  of  the  educational  technologist  is  unclear  and  varies  from  location 
to  location. 

There  are  very  few  professionals  who  actually  hold  the  title  of  "educational 
technologist."    They  are  usually  represented  by  such  titles  as:  media 
specialist,  media  coordinator,  or  library  media  specialist;  sometimes  they 
are  the  director,  supervisor,  or  coordinator  of  educational  media, 
instructional  media,  or  communications.     Newer  and  more  spe':ific  titles  are 
Q  emerging,  including  microcomputer  coordinator,   instructional  computer 
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teacher,  or  specialist  in  educational  computing.    Although  it  is  now  likely 
that  most  large  schools  and  school  districts  have  one  or  more  persons  who 
are  responsible  for  the  administrative,  logistic,  and  instructional  aspects 
of  instructional  media  (or,  in  some  cases,  just  computers),  tliese 
professionals  are  assuming  such  roles  from  a  variety  of  previous  positions 
and  with  varying  types  of  education  and  experience.    The  question  that  seems 
to  underlie  all  of  the  ambiguity  is,  '*What  competencies  are  required  of 
individuals  who  are  designated  to  be  the  educational  technologists?" 

Case  studies  serve  as  models  to  follow  in  the  implementation  of  educational 
technology  applications. 

reople  who  study  the  adoption  of  educational  innovations  know  that  one  of 
the  most  powerful  factors  affecting  adoption  is  evidence  that  an  innovation 
ha.s  worked  in  a  situation  similar  to  the  one  where  it  is  being  considered. 
Although  they  do  not  carry  much  information  in  the  way  of  theory,  research, 
or  development,  case  studies  serve  as  "lighthouses"  or  "pilots"  for  other 
institutions  or  organizations.    The  value  of  case  studies  in  the 
organization  and  management  of  educational  technology  is  demonstrated  in  the 
Office  of  Technology  Assessment  report.  Power  On!  (1988).     Spread  throughout 
this  document  are  29  comprehensive  case  studies  of  technology  use  in 
schools.     Such  reports  as  New  York  State  Teacher  Resource  Centers  and 
Electronic  Networking,  Writing  by  Hand/Writing  with  a  Wordprocessor ,  and 
Software  Evaluation  in  California  help  educational  practitioners  see  how 
others  have  successfully  used  technology  to  solve  specific  problems  of 
teaching  and  learning. 

The  field  of  educational  technology  is  concerned  about  its  status  as  a 
profession. 

Much  of  the  journal  literature  is  about  educational  technology  as  a 
profession,  discussing  such  issues  as  status,  ethics,  legal  aspects, 
history,  and  future  developments  of  the  field.     It  is  obvious  that 
practitioners  of  educational  technology  are  concerned  about  their 
professional  development  and  identity.     They  are  attempting  to  understand 
who  they  are,  what  they  should  be  doing,  and  how  others  view  them.  Such 
concerns  are  typical  among  individuals  who  feel  that  they  are  in  an  emerging 
profession  without  the  tradition  of  an  established  discipline.     It  is  a 
generally  healthy  trend  which  will  probably  be  evident  for  many  years  to 
come . 

Educational  technology  prir.ciples,  products,  and  practices  are  just 
beginning  to  be  integrated  into  courses  and  curricula. 

The  history  of  media  in  education  is  one  of  enrichment  or  enhancement.  Not 
surprisingly,  some  of  the  literature  examines  the  ways  in  which  educational 
technology  and  media  specialists  provide  support  to  teachers/ instructors  to 
improve  their  effectiveness.     There  is,  however,  increasing  interest  in 
media  specialists  ^     curriculum  consultants.    Komoski  (1987)  argues  that 
schools  must  take  ti\e  initiative  and  begin  designing  curricula  that  will 
provide  teachers  and  students  with  a  variety  of  options  and  strategies  for 
Q  achieving  curriculum  goals.     To  this  end,  he  describes  the  Integrated 
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Instructional  Information  Resource,  a  group  of  broadly  accessible, 
electronically  searchable,  and  interrelatable  databases  that  are  designed  to 
assist  educators  in  developing  "opened-out"  curricula.     Information  Power 
(AASL  &  AECT,  1988)  states  that  the  library  media  specialist  must  develop  a 
new,  multi-faceted  role  as  information  specialist,  teacher,  and 
instructional  consultant,  and  offers  guidance  for  assuming  these  roles. 
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A  Hypermedia  Lesson  about  1 875- 1 885  Costume:  Cognitive  Style. 
Perceptual  Modes.  Anxiety.  Attitude,  and  Achievement 

Cognitive  Style 

Cognitive  style  is  the  person's  characteristic  manner  of  receiving,  processing, 
storing,  and  retrieving  information  ( Messick ,  1 969).  Cognitive  styles  are  consistent 
individual  differences  in  mentally  organizing  and  processing  both  information  and 
experience. 

One  of  the  problems  associated  with  the  teaching-learning  process  is  related  to 
determining  how  each  learner  acquires,  stores,  and  recalls  information  effectively. 
Knowing  an  individual's  learning  style  can  be  useful  if  teachers  and  students  are  aware 
of  a  student's  weaknesses  and  strengths.  Through  training,  a  student's  cognitive  style 
can  become  modified  and  adapted  (Keefe,  1985).  Instruction  can  help  individuals 
augment  their  use  of  perceptual  modes.  Perceptual  alternatives  such  as  visual  images, 
audible  statements,  and  verbal  text  may  affect  learners'  abilities  to  assimilate,  retain, 
and  use  information. 

CognitivG  style  studies  have  historically  been  thought  to  have  little  practical 
importance  for  instructional  designers  and  teachers.  This  is  primarily  because  the 
teacher  is  faced  with  the  need  to  design  individual  instruction. 

Individual  instruction  is  feasible  with  the  use  of  technology.  The  use  of 
hypermedia  has  the  potential  to  meet  individual  learning  style  needs  within  one 
program.  HyperCard  Macintosh  programming  allows  Information  to  be  presented  in 
various  forms  as  text,  graphics,  motion,  animation,  voice,  and  still  frames. 

Learning  Style  Instruments 

Researchers  have  attempted  to  measure  students'  information-processing  habits 
by  using  learning  style  tests.  Generally,  these  Instruments  have  been  quite  limited  In 
their  ability  to  measure  these  complex  behaviors.  For  instance,  the  Group  Embedded 
Figures  Test  (GEFT),  developed  by  Witkin  ( 1 950)  has  been  widely  used  to  assess  one 
cognitive  dimension:  field  Independence  vs.  dependence  (Keefe,  1987).  One- dimensional 
learning  style  instruments  led  to  the  development  of  multi-dimensional  learning  style 
tests,  providing  a  more  comprehensive  assessment  of  learning  style.  A  multi- 
dimensional test,  the  National  Association  of  Secondary  School  Principals'  (NASSP) 
Learning  Style  Profile  ( LSP)  includes  twenty-three  independent  constructs  within 
three  areas:  cognitive  style,  affective  style,  and  physiological  style  (Keefe  and  Mor,'- , 
1988).  Within  the  cognitive  component,  nine  generoi  information-processing 
dimensions  are  included:  analytic,  spatial,  discrsninatlon,  categorization,  sequential, 
memory  skills,  visual  perception,  auditory  percfption.  f^nd  emotive  response. 

HyperCard  and  Videodisc 

Instruction  can  be  designed  to  accommodate  individual  differences.  The 
interactive  videodisc  and  computer  system  possesses  some  distinct  characteristics  that 
permit  multiple  modes  of  delivering  information.  The  videodisc  and  computer  lesson  is 
able  to  deliver  information  in  random  order ,  at  a  variable  pace,  with  multiple  sensory 
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learning. 

Instruction  via  videodisc  and  computer  allows  students  to  gather  Information 
from  various  modes  ( visual ,  verbal ,  audible).  HyperCard  programs  enable  learners  to 
control  a  lesson  giving  the  students  the  power  to  choose  the  sequence,  pace,  perceptual 
mode,  and  the  content  of  the  lesson.  HyperCard  has  a  feature  called  buttons  which  are 
sensitive  areas  on  the  page  units  of  the  computer  screen.  When  the  Macintosh  mouse  is 
clicked  on  a  button ,  a  predetermined  action  is  performed,  such  as  showing  a  videodisc 
image,  or  playing  a  pre-recorded  narration.  Since  computers  can  accurately  count  (^h 
selection  chosen  by  a  learner,  computers  are  a  viable  way  to  Investigate  Individuals' 
information-processing  habits. 

An  interactive  videodisc- HyperCard  computer  lfi<^n.  A  Look  Baclcward:  An 
Encounter  with  Late  Victorian  Fashion  was  designed  to  allow  students  to  choose  from 
three  modes  of  receiving  information:  seeing,  hearing,  and  reading.  The  computer 
screen  shows  the  text  and  the  commands  that  allow  the  learner  to  gain  access  to 
information.  A  second  screen,  a  TV  monitor ,  displays  a  video  image. 

The  current  research  attempted  to  discover  which  perceptual  modes  students  used 
to  receive  new  information.  Written  words,  visual  Images,  and  spoken  words  are 
included  In  the  study. 

Computer  Attttude 

Keefe  ( 1 979)  defined  affective  styles  as  motivational  processes  that  are  the 
learners*  typical  modes  of  directing  and  sustaining  behavior.  A  computer  attitude  test. 
Beliefs  About  Computers  Scale  (BAGS),  was  developed  by  Ellsworth  and  Bowman 
( 1 982).  Computer  attitude  referred  to  a  person's  feelings  about  societal  use  and 
individual  use  of  computers.  The  test  consisted  of  1 7  statements  which  are  rated  on  a 
Likert  scale  of  one  to  six. 

Computer  Anxiety 

Computer  anxiety  is  defined  as  the  fear  or  apprehension  felt  by  an  individual 
when  using  computers,  or  when  considering  the  possibilities  of  using  a  computer 
(Simonson.et.  a1.,  1987).  AComputer  Anxiety  Index  (CAIN)  was  designed  to  identify 
students  who  had  computer-related  anxieties,  so  that  steps  could  be  taken  to  reduce 
anxiety  and  improve  achievement. 

Maurer  and  Simonson  ( 1 984)  reported  that  an  individual  with  high  anxiety 
would  avoid  computers,  use  computers  with  great  caution,  and  talk  negatively  about 
computers  and  computing.  AccordingtoSimesandSirkyC  1985),  some  people 
experience  stress  during  an  interactive  lesson.  Anxiety  affects  performance  in  cognitive 
processes. 

Purpose 

The  purpose  of  this  study  was  to  obtain  students'  cognitive  style  profiles  and 
obtain  a  record  of  students'  use  of  three  Information  modes  during  a  hypermedia  lesson, 
This  study  examined  relationships  among  1 )  nine  dimensions  of  cognitive  style  as 
measured  by  a  standarized  learning  style  test,  2)  students'  use  of  three  Information 
modes  (visual  Images,  written  text,  and  audible  description'^^  during  a  hypermedia 
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lesson,  3)  computer  attitude,  4)  computer  anxiety,  ana  5)  achievement. 
Research  Questions 

1 .  What  are  the  dominant  dimensions  of  cognitive  learning  style  for  the  total 
sample  and  two  subgroups:  college  students  in  teacher  education  classes  and 
textiles  and  clothing  classes? 

2.  What  are  the  relationships  among  these  variables: 

i )  cognitive  learning  style,  2)  use  of  information  modes  In  a  hypermedia 
program ,  3)  computer  anxiety,  4)  computer  attitude,  5)  prior  knowledge, 
and  6)  achievement? 

Research  Design 

A  descriptive  and  correlational  design  was  used  for  the  study.  Fifteen  /ar  iables 
were  examined.  Including  cognitive  style  dimensions,  prior  knowledge  and  achievement 
of  1 875- 1 835  costume,  computer  anxiety  and  computer  attitude  ( both  before  and  after 
the  lesson). 

Subjects  were  seventy-nine  undergraduate  students  enrolled  In  Junior  year 
classes:  textiles  and  clothing,  and  teacher  education.  Forty-two  students  were  in  the 
textiles  and  clothing  class  and  thirty-seven  students  were  in  the  teacher  education  class. 

Five  instruments  were  used  to  collect  data:  i )  NASSP  Learning  Style  Profile.  2) 
A  Look  Backward;  Victorian  Fashions,  a  hypermedia  program,  3)  CAIN,  4)BACS,and5) 
achievement  tests.  Valid  and  reliable  measures  were  used. 

Students  completed  the  NASSP  Learning  Style  Profile  test  which  was  used  to 
assess  cognitive  and  perceptual  dimensions  of  learning.  These  dimensions  diagnosed 
strengths  and  w&akness  of  each  Individual's  general  approach  to  processing  Information, 
The  test  consisted  of  cognitive  tasks  to  complete  rather  than  Likert  scale  ratings.  The 
performance  on  the  NASSP  Learning  Style  Profile  was  compared  tu  their  performance  in 
the  hypermedia  program. 

The  perceptual  response  subscales:  visual ,  auditory,  and  emotive  measure 
students'  initial  reectlonstoa  list  of  20  words  by  mentally  "seeing"  a  picture  (visual); 
by  mentally  "hearing"  the  word  (auditory);  or  by  &3ns1ng  an  emotive  (emotive).  The 
emotional  response  may  involve  a  physical  reaction ,  such  as  taking  notes,  or  an  initial 
positive  or  negative  attitude  toward  the  word.  The  relationship  between  cognition  and 
emotion  is  being  examined  by  researchers  since  the  "meaningfulness"  of  learning  may  be 
connected  with  affective  and  psychological  aspects. 

Each  subject  scheduled  a  one-hour  session  to  use  the  hypermedia  lesson,  A  Look 
Backward:  Victorian  Fashions.  The  HyperCard  computer  and  videodisc  lesson  enabled  the 
subjects  to  choose  among  three  Information  modes:  visual ,  verbal ,  and  auditory 
information.  Visual  images  were  available  in  both  line  drawings  and  detailed 
Illustrations  from  a  videodisc.  Verbal  descriptions  about  each  costume  were  written  and 
read  on  the  computer  screen.  Audible  descriptions  of  each  costume  could  be  heard 
through  the  speakers  oi  the  Macintosh  computer.  Objectives  were  given  and  the  students 
chose  modes  of  information  to  accomplish  them. 

An  important  feature  of  the  hypermedia  lesson  was  its  ability  to  track  students' 
progress.  HyperCard  counted  tne  students'  choices  made  during  the  program ,  1 )  visual 
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im?nes,  2)  text  descriptions,  ana  6)  audio  descriptions. 

Students  completed  tests;  1 )  the  Computer  Anxiety  Index  (CAIN),  Beliefs  About 
Computer  Scale  ( BACS),  and  3)  achievement  tests.  These  tests  were  given  before  and 
after  the  students  used  the  hypermedia  program.  The  CAIN  test  was  selected  to  measure 
computer  anxiety.  The  twenty-six  item  survey  is  reliable  and  valid  (Simonson  et  al. , 
1987).  Norm?3tive  data  was  available,  the  average  score  was  60.23  (Simonson  et  al., 
1987) 

The  computer  attitude  test,  BACS,  ( Ellsworth  &  Bowman,  1 982)  was  used  also. 
It  had  a  reliability  of  0.77. 

Results 

Descriptive  statistics,  t-tests,  and  correlations  were  calculated.  T-tests  were 
used  to  compare  the  differences  between  students  enrolled  in  the  two  classes.  The 
Pearson's  product  moment  correlation  was  used  to  identify  relationships  between 
variables. 

Results  and  Interpretation  Related  to  Research  Question  1 

Students'  received  a  score  for  each  of  the  nine  cognitive  style  dimensions  as 
measured  by  the  NASSP  Learning  Style  Profile  (see  Table  1 ).  The  subjects  average 
scores  showed  analytic,  spatial ,  and  categorization  skills  as  the  strongest.  The  weakest 
skills  were  audio  respond  and  discrimination  skills.  Nine  t-tests  wre  calculated  to 
determine  differences  between  the  teacher  6d:ication  and  textiles  and  clothing  classes' 
scores.  Significant  differences  were  found  for  two  dimensions:  sequential  processing 
and  emotive  The  teachor  education  class  had  stronger  sequential  processing  skills  than 
the  textiles  and  clothing  class.  However,  the  textiles  and  clothing  class  had  stronger 
emotive  responses  than  the  teacher  education. 

Pearson  correlation  coefficients  were  calculated  for  each  of  the  nine  cognitive 
dimensions  (see  Table  2)  and  the  results  did  not  indicate  that  these  dimensions 
Gupiicated  one  another.  The  results  signify  that  the  nine  dimensions  are  independent 
constructs. 

Results  and  Interpretation  Related  to  Research  Question  ^2 

Descriptive  Statistics 

Information  nnodes    Students  chose  amnng  three  modes  ( visual  images,  auditory 
statements,  and  written  descriptions)  in  order  to  receive  information  while  using  the 
hypermedia  program.    .  he  computer  counted  the  students'  choices  of  each  of  these 
information  modes.  The  frequency  counti>  were  cha.nged  to  percentages.  The  results  of 
vln  r  choices  are  given  in  the  following  paragraphs. 
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percentage  for  choosing  visual  images  was  55.89  %  of  the  time.  Visual  images  were 
chosen  more  often  than  either  text  or  audio  modes.  The  teacher  education  class  used 
significantly  ( pv03)  more  visual  images  than  the  textiles  and  clothing  class  (see  table 
3). 

Use  of  Text    Statistical  results  showed  that  students  chose  written  descriptions 
in  the  hypermedia  program  on  an  average  of  29.4435  of  the  total  selections.  No 
significant  differences  were  found  between  the  teacher  education  and  textiles  and 
clothing  students'  use  of  text  in  the  lesson. 

Use  of  Audio    Students  used  audio  the  least  of  the  three  information  modes. 
Audio  was  used  1 4.67  %  of  the  time.  No  significant  differences  occurred  between  the 
two  groups  of  subjects. 

Computer  Anxiety  The  results  of  the  CAIN  were  an  average  score  of  60.25  for  the  fir'it 
test  and  59.53  for  the  re-test.  The  analysis  indicated  significant  staiistical  differences 
(p<.0003)  between  the  teacher  education  and  textiles  and  clothing  classes'  level  of 
anxiety.  Little  change  occurred  between  the  first  and  second  administration  of  the  test. 
The  first  test  averages  were  5 1 .86  and  67.47  for  teacher  education  and  textiles  and 
clothing,  respectively.  A  higher  score  indicated  a  higher  level  of  computer  anxiety. 
These  scores  were  compared  to  the  norm  score  62,33  ( Simonson  et  a1. ,  1 987).  The 
textiles  and  clothing  class  seemed  to  experience  more  computer  anxiety  than  the  norm 
group  and  the  teacher  education  class. 

Onmniitftr  Attitude    The  computer  attitude  test,  BAGS  (Ellsworth  and  Bowman,  1982), 
was  given  before  and  after  the  hypermedia  program.  The  time  lapse  between  the  tests 
was  less  than  two  weeks.  No  significant  change  occurred  between  test  scores.  The  re 
-test  average  score  was  41 .5 1 .  A  low  score  indicated  a  more  positive  attitude  toward 
computers  th^n  did  a  high  score.  The  possible  range  of  scores  was  from  1 7  to  1 02. 

A  statistically  significant  difference  (p<.01 )  was  found  between  the  groups 
(average  scores  of  38.57  and  44. 1 7  respectively  for  teacher  educa'don  and  textiles  and 
clothing)  Textiles  and  clothing  students  had  the  higher  average  score,  indicating  that 
they  had  a  less  favorable  attitude  toward  computers. 

Achievement    Achievement  was  considered  secondary  in  importance  to  the  choices  of 
information  modes  examined  in  this  study.  The  erfissss  of  learning  was  of  greater 
interest  to  this  study  than  the  achievement.  The  achievement  test  measured  learning 
that  occurred  within  20  minutes.  The  time  was  limited  because  of  administrative 
reasons  and  also  to  encourage  students  to  be  efficient  in  learning  (choosing  most 
effective  informational  modes  for  their  learning  style).  Within  this  limited  time, 
students  increased  their  scores  from  47%  to  63^  even  though  more  time  was  probably 
needed  for  highe.-  levels  of  learning  to  occur.  No  significant  differences  were  found 
between  the  students  from  the  two  groups'  average  scores  on  the  pretest  or  the  posttest 

Correlation  Analysis 

Pearson  correlation  coefficients  were  calculated  for  each  of  the  tests. 
Correlation  coefficients  were  computed  to  determine  whether  a  relationship  existed 
between  scores  for  each  variable. 

Results  of  the  correlation  an  ysis  showed  significant  positive  correlations  that 
had  p-value  of  .0 1  or  less  ( refer  to  Table  4),  Probably  the  two  most  important 
statistically  significant  correlations  are  as  follows.  The  analytic  skill  ( measured  by  the 
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NASSP  Learning  Style  Profile)  and  text  mode  scores  ( measured  by  hypermedia 
program)  correlated  (r=  33,  p<.01).  Also,  analytic  skill  correlated  with  achievement 
(r=  42,  p<.01). 

This  means  that  a  slight  relationship  between  students  with  stronger  analytic 
skills  and  their  use  of  text  du'^ing  a  hypermedia  lesson.  In  an  analysis  of  variance  test 
It  was  found  that  students  with  strong  analytic  skills  were  more  likely  to  use  text  than 
the  students  with  average  analytic  skills. 

In  general ,  the  higher  the  analytic  score,  the  higher  the  achievement  score.  An 
analysis  of  variance  was  conducted  on  achievements  scores  for  three  groups  of  analytic 
skill  (strong,  average,  weak).  There  was  a  significant  interaction  between  the  scores. 
Students  with  strong  analytic  scores  were  more  likely  to  have  higher  achievement 
scores  than  students  with  average  analytic  scores. 

Conclusions 

These  findings  have  implications  that  are  especially  important  in  educational 
settings.  The  data  demonstrate  that  students  had  individual  cognitive  styles  and 
preferences  for  progressing  through  a  lesson.  The  lack  of  more  significant  relationships 
between  variables  signifies  tnat  there  is  much  variability  in  the  way  siu'  ents  learn. 
Textiles  and  clothing  students  had  fewer  significant  correlations  than  teacher  education 
This  means  that  their  pattern  of  learning  is  harder  to  detect.  Perhaps  this  supports 
instructional  design  that  allows  students  to  choose  from  many  types  of  information 
modes. 

Identifying  learning  styles  is  important  in  order  to  provide  for  the  needs  of  a 
diverse  student  population.  To  identify  students'  learning  styles,  select  reliable  and 
valid  learning  style  instruments.  The  NASSP  Learning  Style  Profile  seemed  to  be  valid 
for  college  students,  although  normative  data  was  available  only  for  college  freshman  and 
not  for  junior  and  senior  year  stur  its.  I  ncluded  in  the  nine  cognitive  dimensions  were 
three  perceptual  modes  (visual  response,  audio  response,  and  emotive).  The  NASSP 
Learning  Style  Profile  derived  its  perceptual  mode  test  from  a  more  Indepth 
instrument,  the  ELSIE  test  (Reinert.  1976).  In  the  future  I  would  suggest  using  the 
ELSIE  test  for  adults.  It  is  a  reliable  and  valid  test  for  measuring  how  language  is 
learned  and  conceptualized.  Since  this  test  more  be  more  thorough  in  assessing  the 
students'  perceptual  responses,  perhaps  a  significant  correlation  would  occur  between 
the  use  of  information  modes  that  they  use  in  a  lesson. 

In  this  study,  three  information  modes  (visual  Images,  text  descriptions,  and 
audio  statements)  were  aval  lable  to  students  as  they  worked  through  a  hypermedia 
lesson  These  information  modes  were  a  measure  of  what  type  of  for  mat  was  used  most 
often  in  learning  about  historic  costume.  This  performance  measure  was  used  to 
examine  learning  style  consistency.  The  visual  mode  assessment  appeared  to  be 
consistent  between  two  scores:  1  )the  visual  response  score  of  the  NASSP  Learning  Style 
Profile  and  2)  the  percentage  of  visuals  used  in  the  hypermedia  lesson,  in  the  NASSP 
Learning  Style  Profile,  students  visual  response  score  was  the  highest  average  of  the 
three  modes.  Similarly,  students'  used  visual  images  most  often  while  using  the 
hypermedia  lesson.  However,  in  the  correlation  analysis,  the  two  measures  did  not 
correlate  significantly .  In  fact,  no  significant  relationships  wpre  found  for  the 
correlations  of  the  three  perceptual  modes  ( v  .  ^i ,  audio,  and  emotive  response)  as 
measured  by  the  NASSP  Learning  Style  Profile  and  the  three  process  variables  as 
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measured  by  the  hypermedia  lesson. 

Possible  reasons  for  the  lack  of  significant  relationships  between  the 
performance  in  the  hypermedia  lesson  and  the  performance  of  three  dimensions  in  the 
NASSP  Learning  Style  Profile  were  as  follows:  1 )  the  exploratory  nature  that  students 
ma/  have  used  to  approach  the  hypermedia  lesson,  2)  the  lack  of  credit  toward  a  course 
grade,  3)  the  diagnostic  level  rather  than  indepth  level  instrument  of  the  NASSP 
Learning  Style  Profile  to  measure  visual ,  audio,  and  emotive  response,  and  4)  students' 
perceptual  response  scores  on  the  NASSP  Learning  style  profile  were  "average", 
although  the  visual  response  score  was  a  "strong-average"  and  the  audio  response  score 
was  "weak-average". 

Students'  profile  of  nine  cognitive  learning  skills  showed  "average"  ratings  for 
all  nine  dimensions,  although  some  skills  were  "weak-average",  "mid-average",  and 
"strong- average".  Further  research  can  investigate  the  Interaction  of  individual's 
cognitive  scores  and  students'  choice  of  Information  modes,  attitudes,  and  achievement. 
The  sample  should  be  stratified  sample  consisting  of  an  equal  number  of  students  in 
three  groups  ( "weak",  "average",  and  "strong")  for  each  of  t^e  nine  cognitive  skills. 
Statistical  tests  could  analyze  the  difference  between  the  grc  n  means  (weak,  average, 
strong)  and  indicate  if  there  is  an  interaction  between  the  dependent  and  indep  ^^ndent 
variablp. 
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Abstract 

Research  in  the  field  of  computer-based 
instruct  ion  has  led  to  conflicting  evidence  of  the 
efficacy  of  learner  control  over  ins true t ion , 
Typical   implement at  ions  of  learner  control   in  CBI 
allow  learners  to  proceed  at  their  own  pace, 
control   the  sequence  of  instruct  ion,   choose  the 
amount  of  practice,   or  decide  the  level  of 
difficulty  attempted.     Benefits  of  learner  control 
have  been  reported,   primarily  in  the  affective 
domain.     Results  have  been  mixed,   however,  in 
achievement  when  the  learner  is  allowed  cont ro 1 
over  the  amount  and  sequence  of  instruction. 
Recent  research  fitudies  allowing  the  learner  to 
manipulate  contextual  variables  such  as  text 
density  have  demonstrated  learner  control  to  be  a 
viable  option  in  CBI.     The  imp li cat  ion  arises  that 
those  variables  which  do  not  manipulate  the  amount 
of  content  support  are  those  over  which  the 
learner  should  be  allowed  control.     This  paper 
discusses  five  learner  control  variables  and 
suggests  recommendations  for  implementing  these 
variables  in  CBI  3oftware  and  research. 
Thirty-one  references  are  included. 
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Russian  sclent  is is  successful ly  launched  an 
artificial  satellice  in  1957  which  they  named 
Sputnik,   galvanising  Western  society  into  a 
unified  effort  to  surpass  the  Soviets.  Our 
relatively  tardy  entrance  into  the  race  for  space 
was  attributed  to  a  dearth  of  qual ified  personnel 
in  our  laboratories  and  classrooas^   and  this  lack 
was  in  turn  attributed  to  some  fau'^t   in  the 
education  of  our  youth.     New  methous  were  tried 
and  discarded .     Every  aspect  of  educat  ion 
underwent  scrutiny  in  our  efforts  to  improve  the 
qual i ty  of  our  schools.     Since  1957,  educators 
have  been  subjected  to  continued  pressure  not  only 
to  improve  the  quality  of  our  schools  but  also  to 
educate  a  broader  segment  of  society.   One  response 
to  this  pressure  was  the  development  of  systems 
for  individual   instruct  ion  based  on  Skinner* s 
theory  of  behaviorism   (1938)   and  Bloom's  (1971) 
theory  of  mastery  learning.      Instructional  systems 
from  Programmed  Instruction  to  computer-based 
instruction  have  been  designed  to  individualize 
instruct  ion . 

Carrier   (1984)   observes  that  no  other 
delivery  system  except  the  teacher  equals  the 
computer  in  its  potential  ability  to  accommodate 
the  needs  of  individual   learners.     The  use  of 
computers   in  education  has  finally  made 
individualization  a  feasible  goal.      In  designing 
individual   instruction  the  question  has  become  to 
what  extent   learners  control   their  own  instruction 
rather  than  submit  to  computer  control.   This  paper 
identifies   five  classes  of  learner  control 
variables,   discusses   the  effectiveness  of  each, 
proposes  guidelines  for  application  of  each 
variable,   and  suggests  directions  for  future 
research . 

Learner  control.     Before  we  can  categorize 
research  in  learner  control  we  must  have  a  working 
definition  of  the  term  itself.   There  has  been 
cons ider able  debate  over  what  constitutes  control, 
from  allowing  the  student  to  make  decisions  over  a 
single  aspect  of  instruction  to  almost  complete 
control  over  a  whole  course  of  study  (Merrill, 
1984).     Sasscer  and  Moore   (1984)   state  that  "the 
research  literature  related  to  learner  control  of 
instruction   is  characterized  by  reports  of 
contradictory  findings  and  equivocal  terminology** 
(p. 28).     According  to  McCann   (1981)   the  issue  has 
been  reduced  to  an  "all  or  none"  proposition.  He 
points  out,   however,   that  education  itself 
progresses  from  the  kindergarten  with  scarcely  any 
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learner  con t rol  to  graduate  school  where  learner 
control  of  study  predominates.     In  the  broadest 
sense ,   therefore ,   we  can  define  learner  control  as 
student  decision-making.     Student  decision-making 
in  strategy,   pe^^uence,   pace,   amount  of  practice, 
and  level  of  difficulty  have  all  been 
investigated. 

Sequence.     Judd,   Bunderson,   and  Bessent 
(1970)   studied  the  effects  of  four  levels  of 
learner  control  in  a  remedial  mathematics  course 
for  college  students:    (1)   computer  control; 
(2)   learner  control  over  sequence;    (3)  student 
control  over  amount  of  practice;   and  (4)  complete 
learner  control.     They  found  that  students  who  had 
done  poorly  on  pretests  performed  worse  under 
learner  con t ro 1 .     They  also  noted  that  when 
students  were  allowed  control  of  sequence  some 
topics  were  omitted  which  might  have  been  crucial 
to  understanding.     Further,   students  did  more 
poorly  with  complete  control  over  their  own 
instruction  than  those  who  were  under  complete 
computer  control.     In  utilizing  the  TICCIT  system, 
which  allows  student  control  of  course  flow, 
Bunderson   (1976)   found  that  there  was  no 
difference  in  overall  achievement  between  these 
students  and  those  in  regular  classrooms.  He 
pointed  out,   however,    that  some  students  make  poor 
strategy  choices  when  left  with  no  instructional 
advice.   TICCIT  is  reported  as  a  learner  control 
system,   with  program  mechanisms  to  allow  the 
student  to  select  and  sequence  content.     One  of 
TICCIT's  designers  stated  (Merrill , 1979)     that  if 
he  were  designing  a  second  version  he  would 
include  more  help  for  the  student  to  know  how  to 
proceed .     Laur il lard  ( 1984)   found  that  students  do 
make  use  of  instructional  suggestions  and  like 
being  given  advi ce  on  what  to  do  next   in  terms  of 
sequence  and  strategy.   Using  the  PLATO  system, 
Lahey,   Crawford,   and  Hurlock  (1975)   also  found 
that  there  was  no  significant  difference  in 
performance  between  students  who  had  control  over 
flow  and  those  who  did  not,   although  the  learner 
control  group  spent  more  time  on  task  than  the 
autonomous  group  even  though  both  groups  were 
self -paced.     Seidel ,   Wagner,   Rosenblatt , 
Hillelsohn,   and  Stelzer  (1978)   found  that  not  all 
students  used  good  learning  strategies  when  given 
control  of  sequencing  in  a  programming  course  in 
Cobol.         Jacobson  and  Thompson  (1975)  interpreted 
learner  control  as  student  self-teaching,   and  as 
such  proposed  a  sort  of  "teacher  training"  for 
students  in  order  to  maximize  their  strategy 
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decisions*     Derry  (1984)   also  investigated 
strategies  training  in  preparing  students  for 
control  of  instruction.     She  concluded  that  no 
single  instructional  device  is  known  that  will 
greatly  enhance  learning  effectiveness,  with  the 
possible  exceptions  of  what  she  teraed  "forced 
practice"  and  feedback. 

Pacing.     Another  aspect  of  investigation  in 
learner  control  has  been  the  pace  of  p res en  tat  ion 
of  material .     Even  in  this  area  there  has  not  been 
clear  consensus .     Gay  ( 1986)   found  that  high 
aptitude  students  perforBied  better  than  low 
aptitude  students  when  given  control  over  pace. 
Goetzfried  and  Hannafin   (1985),   however,    found  no 
di  f f erences  in  achievement  among  students  in 
treatments :    ( 1 )   adaptive  control   (external  based 
on  accuracy  of  response);    (2)    learner  control  with 
advisement;    and   (3)    linear  control   (no  control 
except  over  pace).     The  conclusion  then  would  be 
that   if  there  is  no  difference  in  achievement  the 
treatment  of  choice  would  be  that  which  took  less 
time,    the  linear  control   treatment.     One  area  of 
interest   in  research  on  pace  of  instruction  is 
student  procrastination.     Reiser   (1984)  studied 
the  effects  of  three  pacing  procedures:  (1) 
rewards   for  comp let ing  on  time;    (2)  penalty  for 
tardy  completion;   and  (3)   learner  control  of 
pacing.     He  found  that  there  was  no  difference  in 
achievement  among  the  groups,   but  that   the  learner 
control  group  only  met  22.2%  of  deadlines  compared 
to  ^l.^X  met  by  the  penalty  group  and  43. 8\  by  the 
reward  group.     There  was  no  significant  difference 
in  attitudes  concerning  the  course  or  in 
withdrawal  rate. 

Ins  tract i on al  support.       If  we  can  find  no 
consensus  for  sequencing  or  pacing,   there  is  some 
agreement  over  student  control  of  number  of 
examples.     Again,   studies  show  no  significant 
differences  in  achievement  among  groups  who  have 
varying  amounts  of  control  over  examples  given 
(Gay,   1986;   Axtell,    1978;   Wilcox,  Richards, 
Merrill,   Chris tensen,   &  Rosenvall,    1978) ,     Wi Icox 
et  al,    (1978)   assigned  college  students  learning 
logarithms  using  the  TIC ;iT  system  to  either  a 
learner  control  group ,   with  choice   in  the  number 
of  examples,   or  to  a  "yoked"  group.  Students 
assigned  to  this  group  were  randomly  paired  with 
students  in  the   learner  control  group  and  required 
to  study  the  same  number  of  examples  as  their 
counterparts   in  that  group.     Since  there  •^'ere  no 
differences  in  posttest  performance  of  the  two 
groups  Wilcox  maintained  that  there  wa3  no 
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evidence  that   learner  control  of  number  of 
exanples  would  accommodate  ind  i vidua 1  differences. 
Carrier,   Davidson,   and  Williams  (1985);  however, 
found  that  high  ability  students  did  better  with 
options  for  the  number  of  examples  than  did  low 
ability  students.     Amount  of  practice  was  another 
variable  studied  along  with  number  of  examples. 
Judd  (1981)   found  that   learner  control  resulted  in 
subjects  studying  fewer  practice  modules,   but  that 
post  test  scores  did  not  differ  from  those  students 
who  had  no  such  control.       Learner  control   of  the 
number  of  pract i ce  examp les  was  f ound  to  be  an 
ineffective  strategy  by  Ross  and  Rakow  (1981)  when 
compared  to  subjects  who  had  adaptive  program 
control  of  instruction.       Montanelli  and  Steinberg 
(cited  in  Steinberg,    1977)   found  that  students  in 
a  computer  science  lesson  followed  practice 
suggestions  on  easy  problems  but  ignored  practice 
suggestions  with  problems  on  more  difficult 
concepts , 

A  premise  of  much  of  the  research  on  learner 
control  has  been  that  students   in  control  of  their 
own  learning  are  better  motivated  than  their 
counterparts  with  no  such  control.     No  differences 
in  attitude  were  found  among  treatment  groups  by 
Lahey  (1978).     Students  responding  to  informal 
questioning  by  Ross  and  Rakow   (1981),  however, 
reported  positive  att  itude  responses  to  learner 
control.     Hazen  (1985)  states  that  learner  control 
over  CAI  choices  along  with  positive  feedback 
could  keep  learners  motivated  when  working  through 
CAI  programs.     Salomon  (1985)   contends  that 
learn ing  depends  on  the  learner's  "mindfulness", 
which  in  turn  depends  on  expectations, 
perceptions,   attitudes,   and  goals.  Mindfulness 
then  is  a  volitional  matter,   the   importance  of 
which  increases  the  more  control  over  the  learning 
process  students  are  given. 

Because  of  the  lack  of  consensus  on  aany  of 
the  aspects  of  learner  control  research  and 
contradictory  results  found  in  various  studies  it 
is  not  possible  to  pronounce  judgement  on  the 
entire  concept  of  learner  control  of  instruction. 
Several   trends,   however,   can  be  deduced. 
Originally,   designers  of  instruction  interpreted 
learner  control  as  the  opportunity  for  students  to 
sequence  content  in  a  particular  course  of 
instruction  and  to  control  pace.      Increase  in  time 
on  task  was  considered  a  price  one  paid  for 
positive  effects  in  the  affective  domain.  Later 
research  has  emphasized   learner  control  of 
strategies  of  instruction,   but  many  researchers 
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are  hypothesizing  that  students  do  not  have  the 
metacogn  i  t  i  ve  skills  to  utilize  the  inost  effective 
strategies.   Indeed,   the  entire  concept  of 
advisement   (Johansen  and  Tennyson, 1983 ;  Mullen, 
1984)    is  predicated  on  the  idea  that  students  need 
help   in  devising  learning  strategies.  Results 
that    indicate  better  results  froa  high  ability 
students   (Carrier  et  al.,    1985)   suggests  that 
these  saffle  students  have  the  necessary 
metacogn i t i ve  skills  already  in  place  and  that 
their  superior  performance  is  a  result  of  this 
acqui  sition  of  skills  rather  than  control  of 
instruction.       As  an  alternative  approach  in 
recearch,   therefore,    it   is  suggested  that 
variables  less  sensitive  to  lack  of  wetacogni  t  ive 
strategies  be  investigated. 

Context .       One  such  area  with  potential  for 
research  would  be  learner  control  over  the  context 
in  which  examples  are  presented.     Anand  and  Ross 
(1987)   studied  the  use  of  personalized  word 
problems   in  mathematics  for  fifth-  and  sixth-grade 
children.     Problem  contexts  were  varied  by 
altering  the  referents  and  background  themes  of 
example  problems  while  keeping  the  numerical 
values  and  measurement  units  constant. 
Personalized  contexts  used  personally  familiar 
items  obtained  from  biographical  questionnaires. 
Findings  indicate  the  personalized  materials  to  be 
beneficial  across  a  variety  of  learning  outcoaes 
as  well  as  for  attitudes  toward  the  task.  Based 
on  the  idea  that  older  students  appear  to  have 
strong  individual  preferences  for  general  themes 
,e.g.   education,   nursing,   business,  (Ross, 
McCormack,   &  Krisack,    1986),   a  study  allowing 
college  students  control  of  background  theme  in  a 
statistics  lesson  (Ross,   Morrison,   and  0 • Del 1 , 
1988)   indicated  that  students  pick  a  variety  of 
themes   instead  of  remaining  with  only  one  theme 
throughout   the  lesson.     They  reported  most 
subjects  varied  their  selections  across  lessons, 
suggest  ing  some  at  tempt  by  subjects  to  adjust 
strategies  as  perceived  needs  changed. 

Another  contextual  variable  of  CBI  over  which 
the  learner  could  exercise  control   is  the  text 
dens  ity  level  of  the  material  presen ted . 
Morrison,   Ross,   and  O'Dell   (1988)  presented  the 
narrative  text  of  a  statistics   lesson  containing 
low-density  (concise)   text,  high-density 
(conventional)   text,   or  learner-controlled  density 
1  eve  1.   Each  version  contained  the  same 
information,     but   the  low-density  version 
eliminated  details  and  nonessential  words. 
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resulting  in  an  approximately  50%  reduction  in 
both  number  of  words  and  screen  area  required. 
Findings  imply  adaptive  uses  of  learner  control 
and,   as  such,    ".•.reinforce  the  idea  that  learner 
control  is  not  a  unitary  concept ,   but  rather  a 
collection  of  strategies  that  work  in  different 
ways  depending  on  what   is  being  controlled  by 
whoa**   (Morrison,   Ross,   O^Dell,   &  Schultz,  1988, 
p. 73).     In  these  ways  students  maintain  control  of 
contextual  variables  with  instructional  support 
controlled  by  the  program.     Thus,   students  can 
exercise  control  over  how  they  want  to  learn,  with 
its  accompanying  affective  benefits ,   whi le 
fol lowing  a  lesson  prepared  by  the  prof ess  ion a 1 
designer  who  maintains  control  over  the  amount  of 
instructional   support   the  student  needs. 
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Summary 

Early  research  on  learner  control  focused  on 
content  variab les .     Pacing  left   to   learner  control 
has  been  found   ineffective  because   learners  tend 
to  procrastinate  or  to  exit   lessons  premature ly . 
Sequence,   or  content  flow,   has  been  an  ineffective 
variable   left  under  student  control  since 
appropriate  sequencing  of  the  lesson  requires 
prior  knowledge  of  content.     Number  of  examples 
and  level  of  difficulty  are  also  inappropriate 
decisions   for  the   learner  to  make  because  students 
frequently  lack  pt^tacogn i t i ve  skills  to  make  such 
strategic  decisions.     Advisement  strategies  have 
attempted  successfully  a  diagnostic/  prescriptive 
approach  to  control  of  content  variables.  Current 
research,   however,   has  focused  on  contextual 
variables  under  control  of  the   learner.  Variables 
such  as  text  density  l#»vel  and  background  theme 
have  allowed  learner  control  of  stylistic 
qualities  of  the  lesson  without  sacrificing 
instructional  support.     Future  research  of  learner 
control  in  CBI,   therefore,   should  concentrate  on 
the  continuum  of  contextual  variables  with  their 
accompanying  affective  benefits  rather  than 
content  variables  with  their  inherent 
disadvantages  and   implicit  "all   or  none" 
phi losophy . 
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ENHANCING  MOTIVATION  AND  THE  ACQUISITION  OF 
COORDINATE  CONCEPTS  THROUGH  THE  USE  OF  CONCEPT  TREES 


ABSTRACT 

This  study  asses*^ed  the  effects  of  providing  learners  with  a  graphic  illustration  of 
coordinate  concept  relationships  to  supplement  learning  from  text-based  instruction. 
Seventy-three  undergraduate  students  weiie  given  a  passage  of  approramately  1200 
words  in  length,  describing  Ausubel's  Categories  of  Meaningful  Learning.  Half  of 
the  students  £dso  received  a  graphic  concept  tree  which  illustrated  the  relationship 
between  the  concepts  presented  in  the  text.  Findings  from  analyses  of  variance  on 
an  instructional  motivation  measure  and  the  immediate  post  test  indicate  that 
students  who  used  the  concept  ti'ee  ouq)eiformed  tiiose  learner  who  did  not  use  die 
concept  tree,  and  that  students  who  ustd  the  concept  tree  reported  significantly 
higher  amounts  of  attention,  confidence,  iuid  satisfaction  with  the  instructional 
materials.  No  interactions  v/ere  found  between  use  of  the  concept  tree  and 
vocabulary  ability. 
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ERIC 


ENHANCING  MOTIVATION  AND  THE  ACQUISITION  OF 
COORDINATE  CONCEPTS  TliROUGH  THE  USE  OF  TONCiPT  TREES 

The  use  of  modem  technologies  (e.g.,  CAI,  video,  interactive  videodisc, 
telecommunications,  etc.)  has  seen  a  recent  gain  in  popularity  in  public  and  higher 
education.  However,  significant  amounts  of  "required"  course  information  is  still 
transmitted  though  printed  text.  Umversity  students  spend  much  of  their  academic 
task-oriented  time  reading  handouts,  textbooks,  and  journal  articles.  To  keep  up 
with  class  ai'signments,  students  are  frequently  asked  to  read  ten  to  fifty  pages  of 
printed  text  aaily.  Add  to  tliat  the  rigor  of  class  projects  and  examinations  and 
suddenly,  the  ability  to  efficiently  identify  and  encode  key  concepts  within  printed 
text  becomes  crucial  to  the  academic  success  of  college  students.  Due  to  the 
complex  nature  of  son»  print  materials,  instractors  may  spend  significant  amounts 
of  class  time  reviewing  reading  assignments  If  the  purpose  of  instruction, 
however,  is  to  teach  higher  order  thmking  skills,  it  seems  that  an  instructional  tool 
which  facilitates  the  acquisition  of  key  concepts  would  give  students  and  instructors 
more  time  to  concentrate  on  analysis,  synthesis,  and  problem  solving. 

Textbooks  and  journal  articles  are  also  often  viewed  by  students  as  boring 
or  irrelevant  to  their  needs.  In  particular,  low  ability  readers  may  not  be  confident 
in  their  aptitude  to  identify  key  concepts  and  complete  reading  assignments  with 
sufficient  meaning.  In  turn,  students  who  fail  to  derive  meaning  from  class 
assignments  may  find  instruction  unsatisfying.  Considering  the  demands  of 
stuctent  time  and  the  seemingly  low  motivation  to  read  print-based  materials,  it  is 
not  surprising  to  find  that  students  frequenriy  fall  behind  on  their  reading 
assignments. 

Many  instructional  tools,  study  methods  and  strategies  have  been  designed 
to  promote  reading  comprehension.  However,  little  has  been  done  to  implement 
these  tools  in  college  course.  Less  has  been  done  to  define  the  motivational  impact 
of  these  tools  and  ^eir  relationship  to  cognition.  The  purpose  of  this  study  was  to 
determine  the  effects  of  graphic  advance  organizers  on  reading  comprehension  and 
motivation.  Three  hypotheses  were  tested  in  this  study: 

H 1  The  addition  of  a  concept  tree  to  supplement  text-based  materials  will 
increase  students'  ability  to  acquire  coordinate  concepts  compared  to 
students  receiving  only  text-based  instruction. 

H2  The  addition  of  a  concept  tree  to  supplement  text-based  materials  will 
increase  students*  perceived  level  of  motivation  compared  to  students 
receiving  only  text-based  instruction. 

H3   The  addition  of  a  concept  tree  to  supplement  text-based  materials  will 
increase  students'  perceived  level  of  confidence  in  successfully 
completing  the  reading  assignment  compared  to  students  receiving 
only  text-based  instruction. 
An  additional  research  question  of  interest  was,  "Do  students  with  different 
reading  abilities  experience  differential  gains  in  achievement  when  presented  with  a 
concept  tree  as  a  graphic  advance  organizer?" 

Review  of  Literature 

Over  the  past  two  decades,  significant  amounts  of  research  has  been 
conducted  on  the  effects  of  advance  organizers  (AOs),  graphic  organizers  (GOs), 
graphic  advance  organizers  (GAOs),  and  graphic  post  organizers  (GPOs)  on 
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leading  comprehension,  and  coordinate  concept  learning.  Numerous  studies  have 
also  examined  various  motivational  constructs  and  their  impact  on  tiie  initiation, 
direction,  and  persistence  of  goal  oriented  behavior.  However,  cognition  and 
motivation  have  traditionally  been  studied  in  separate  ccMitext  (Nenniger,  1988). 

AOs.  GOs,  GPQs.  &  GAOs  &  Concept  trees 

Early  studies  with  advance  organizers  (AOs)  yielded  contradictory  results 
(Luiten,  Airies,  &  Ackerson,  1980).  However,  recent  reviews  utilizing  a  variety  of 
analytic  procedures  (e.g.,  meta-analysis,  t-statistics,  voting  technique,  etc.)  indicate 
tiiat  AOs  do,  in  fact,  and  facilitate  learning  (Kozlow,  1978;  Mayer,  1979;  Luiten, 
Ames,  &  Ackerman,  1980;  Stone,  1983).  Although  researchers  agree  that  the 
presentation  of  AOs  may  enhance  learning,  studies  of  such  variables  as  age,  ability, 
and  styles  of  presentation,  have  yielded  mixed  results. 

Barron  (1980)  noted  that  graphic  organizers  (GOs)  were  developed  as  a 
variation  of  Ausubel's  advance  orgaiuzcrs.  GOs  are  defined  as  visual 
representations  which  portray  relationships  among  key  terms  derived  from  the 
learning  task  (Moore  &  Readence,  1984),  Three  studies  reviewed  the  effects  of 
GOs  on  learning  and  retention.  Smith  (1978)  concluded  that  GOs  did  not  facilitate 
learning.  However,  Smith  analyzed  only  six  studies  which  used  GOs  solely  as 
teacher-directed  pre-reading  exercises.  Applying  meta-analytic  techniques  on  16 
investigations,  Moore  &  Readence  (1980)  computed  a  small  positive  overall  effect 
of  GOs  on  learning  (average  effect  size  =  .30,  standard  error  =  .05).  In  a  more 
recent  review  of  research,  Moore  &  Readence  (1984)  found  similar  results;  the  use 
of  GOs  increased  learning  regardless  of  subject  matter  and  other  variables  under 
study  (average  effect  size  =  .22,  SD  =  .58).  Moore  &  Readence  (1984)  also 
concluded  tiiat  grapWc  post  organizers  (i.e.,  GOs  constructed  by  either  the  teacher 
or  tiie  leamer  after  tiie  learning  task)  have  a  greater  effect  on  learning  than  graphic 
advance  organizers  (GAOs).  Graphic  post  organizers,  however,  require 
significantiy  greater  investments  by  instructors  and  students  compared  to  GAOs  in 
terms  of  implementation  and  directed  use. 

One  type  of  graphic  organizer  (i.e.,  concept  trees),  requires  relatively  little 
time  in  terms  of  development  and  implementation  and  has  yielded  consistent  gains 
in  student  performance  on  immediate  posttest  measures  (Tessmer  &  Driscoll, 
1986).  Developed  by  Tessmer  &  Driscoll,  concept  trees  are  defined  as  "...a 
diagrammatic  dsplay  of  tiie  prepositional  relationships  among  concepts.  It  presents 
in  a  h  erarchical  fashion  tiie  superordinate  and  subordinate  class  relationships 
(genus;  and  differentia)  of  a  set  of  related,  coordinate  concepts"  (p.  196).  Unlike 
traditional  GOs  and  AOs  which  related  learning  tasks  to  superordinate  and 
subsuming  concepts  tiiat  have  abeady  been  learned  (St(Mie,  1983;  Estes,  Mills  & 
Barron,  1969),  concept  trees  do  not  necessarily  depict  prior  knowledge.  Concept 
trees  also  regularly  include  brief  narrative  examples  of  key  ideas  (see  fig.  1). 
Altiicugh  concept  trees  have  been  associated  with  increased  achievement  compared 
to  expository  metiiods  of  presenting  information,  the  use  of  concept  trees  as  graphic 
advance  organizers  and  its  resultant  impact  on  student  motivation  has  not  been 
investigated. 

Qualitatively,  Moore  &  Readence  (1984)  indicated  tiiat  tiie  GOs  may  affect 
teachers'  and  students'  motivation.  By  analyzing  the  results,  discussions,  and 
conclusions  of  23  studies,  Moore  &  Readence  noted  tiiat  "teachers  believed  that 
GOs  prepared  tiiem  to  help  students  cope  witii  particular  pieces  of  content"  (p.  15). 
However,  "In  pre-reading  conditions,  many  students  apparentiy  failed  to  see  how 
GOs  were  connected  witii  the  materials  to  be  learned"  (Moore  &  Readence,  1984, 
p.  15).  Thus,  the  use  of  GOs  appears  to  be  motivationally  appealing  but  tiie 
perceived  relevance  of  GAOs  may  be  low  which,  in  turn,  may  cause  students  to  pay 
littie  attention  to  tlie  GAOs.  Although  Moore  &  Readence  provides  some  insight 
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concerning  the  affective  impact  of  G AOs,  empirical  data  describing  the  effects  of 
GAOs  on  student  motivation  is  limited 

Motivation 

Performance  can  be  viewed  as  a  function  of  motivation  and  ability.  Indeed, 
psychologists  have  long  studied  the  effects  of  personality  and  motivation  on  human 
behavior  and  performance.  03nstructs  such  as,  the  Need  for  Achievement 
(Atkinson,  1964;  McClelland,  1976),  Self-Efficacy  (Bandura,  1977),  Attribution 
(Weiner,  1980),  Locus  of  Control  (Rotter,  1954),  Anxiety  (Spiclberger,  1972; 
Endler,  1975),  Hygiene  (Herzberg,  1966),  Pfcrsonal  Causation  (deCharms,  1976), 
etc.,  have  all  been  correlated  to  behavior  and  performance.  In  the  field  of 
instruction,  numerous  authors  have  also  cited  the  importance  of  learner  motivation 
on  performance  (Mager  &  McCann,  1961;  Mager  &  Clark,  1963;  Gagne,  1965; 
Briggs,  1980;  Wlodkowsky,  1981;  Brophy,  1983;  Keller,  1987a).  Studies  by 
Alcterman  &  Mahler  and  Jol  aston  (cited  by  Keller,  1983)  indicate  that  instructional 
programs  associated  with  superior  learning  may  result  in  increased  student  attrition 
and  procrastination  relative  to  a  comparison  group.  This  suggests  that  instruction 
can  affect  learning  and  motivation  as  separate  variables,  which  provides  support  for 
studying  such  effects  separately.  Recent  investigations  have  also  shown  that 
student  performance  may  be  regulated  by  interactions  between  cognitive  variables 
and  the  type  of  motivation  stimulated  by  the  learning  environn»nt  (Pintrich,  Cross, 
Kozma,  &  McKeachie,  1985;  Pintrich,  1987;  Nenniger,  1987).  For  example, 
students  with  contrasting  conceptualizations  of  motivation  (i.e.,  intrinsic,  extrinsic, 
achievement,  and  fear  of  failuiie)  identified  differing  approaches  to  learning 
(Entwistie  and  Kozeki,  1988). 

Although  the  need  to  addreis  motivational  issues  has  been  stressed  (e.g., 
Martin  &  Driscoll,  1984),  systematic  efforts  to  integrate  motivational  requirements 
with  instructional  design  have  been  lacking  (Keller,  1987a:  Reigeluth,  1979). 
Some  theorist  label  "motivation**  as  an  explicit  element  of  their  design  model  (e.g.. 
Bloom,  1976;  Cooley  &  Lohnes,  1976;  Gagne,  1977;  Reigeluth  &  Merrill,  1979). 
However,  systematic  procedures  for  manipulating  motivation  factors  have  not  been 
elucidated  witii  die  rigor  associated  with  concept  acquisition  (Keller,  1983).  As 
Briggs  (1980)  observed,  "our  theories  or  models  of  design  do  not  take  enough 
account  of  motivation,  even  though  we  may  acknowledge  that  motivation  effects  are 
stronger  then  treatment  effects"  (p.  49). 

The  need  for  improved  methods  for  measuring  motivation  and  the  need  for  a 
better  tiieory  upon  which  to  base  these  measures  have  been  discussed  by  Cooley  & 
Lohnes  (1976)  and  Keller  (1983).  However,  direct  measures  correlating 
motivation  with  performance  have  been  rather  elusive  (Keller,  Kelly,  &  Dodge, 
1978).  This  has  been  due,  in  part,  to  the  absence  of  a  comprehensive,  and 
systematic  approach  to  studying  motivation  in  relation  to  schools  of  learning 
(Keller,  1988).  Current  measures  of  motivation  only  take  into  account  a  limited 
number  of  constructs  which  have  been  associated  with  behavior  and  performance. 
Thus,  in  order  to  develop  a  holistic  view  of  learner  motivation,  researchers  would 
have  to  employ  a  battery  of  tests  which  is  not  practical  in  educational  settings. 
Although  multi-dimensional  models  of  motivation  have  been  developed,  few 
provide  educational  practitioners  with  a  practical  method  of  addressing  motivational 
requirements  that  takes  into  account  the  nmge  of  motivational  constructs  which  have 
been  correlated  to  performance. 

Keller  (1987a)  has  proposed  an  eclectic  model  of  motivation  which  provides 
educational  practitioners  with:  (1)  a  typology  to  organize  knowledge  of  human 
motivation,  (2)  a  pragmatic  metiiod  of  measuring  the  motivational  impact  of 
instruction,  and  (3)  a  systematic  procedure  for  designing  motivationally  effective 
instruction.  Based  on  a  rigorous  examination  of  current  motivational  theories. 
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Keller  (1987a)  synthesized  the  ARCS  inodel  which  identifies  four  basic 
motivational  requirements.  Keller  (1987a)  posits  that  in  wder  to  motivate  students 
to  leam,  instruction  must:  (1)  gain  and  sustain  ieamcr  Attention,  (2)  be  Relevant  to 
leamer  ne^,  (3)  foster  learners'  Confidence  in  their  ability  to  successfully 
complete  the  task,  and  (4)  be  Satisfying  to  learners  by  meeting  their  expectations 
and  providing  equitable  feedback  (for  a  more  complete  review  of  the  ARCS  model, 
see  KeUer  1983, 1987a  and  1987b).  Keller  (1988)  and  Keller  &  Subhiyah  (1987) 
have  developed  two  instruments  to  measure  die  motivational  effectiveness  of 
instruction  based  on  the  ARCS  model-the  Instructional  Materials  Motivation  Scale 
(IMMS)  &  the  Course  Interest  Survey  (CIS).  Although  empirical  evidence 
supporting  the  construct  validity  of  tiie  ARCS  model  is  still  rather  limited,  studies 
have  rcpOTted  statistical  reliability  for  its  measures  (Cronbach's  alpha  ranging  from 
.6  -  ^8)  several  different  instmctional  settings.  By  systematically  assessing  both  the 
cognitive  and  motivational  impact  of  instruction,  we  should  be  able  to  gain  a  better 
understanding  of  the  factors  which  contribute  to  leamer  performance  in  specific 
instructional  situations. 

Methods 

Subjects  &  Design 

Subjects  were  male  and  female  undergraduate  education  students  (n=125) 
enrolled  in  3  sections  of  an  educational  psychology  class  at  the  Florida  State 
University  in  Tallahassee,  Florida.  On  the  first  day  of  class,  the  36-item  Advanced 
Vocabulary  Test  n  from  the  Kit  of  Factor-Referenced  Cognitive  Tests  (Ekstrom, 
French,  Harman,  &  Dermen,  1987)  was  administered  to  all  students  as  a  pretest 
measure.  The  internal  consistency  of  the  pretest  was  calculated  as  a  split-half 
reliability  coefficient.  As  corrected  with  the  Spearman-Brown  prophecy  formula, 
the  reliability  coefficient  was  .89. 

Based  on  pretest  scores,  the  matched  pair  technique  (Tuckman,  1988)  was 
used  to  assign  students  to  two  groups*  Following  assignment,  one  group  was 
randomly  chosen  to  receive  tiie  materials  with  the  concept  tree;  the  other  group 
comprised  the  control,  and  received  the  material  without  the  concq)t  tree. 

Due  to  eventual  scheduling  difficulties,  students  in  sccti(Mi  3  of  the  class  were 
unable  to  participate  in  the  experiment*  A  comparison  of  mean  pretest  scores  and 
standwd  cteviations  on  the  experimental  pretest  measure  revealed  no  difference 
between  tiie  three  sections  tiiat  might  systematically  affect  subsequent  results,  so  the 
loss  of  students  in  section  3  was  not  considered  a  threat  to  the  internal  validity.  The 
exclusion  of  the  students  in  section  3  and  subsequent  attrition  due  to  students 
dropping  the  course  or  being  absent  on  the  day  of  tiie  experiment  reduced  the  final 
subject  pool  to  73. 

Instructional  Materials 

A  common  set  of  instmctional  materials  was  developed  for  both  treatment 
and  control  groups.  This  included  a  printed  1323  word  passage  describing 
different  kinds  of  meaningful  learning  (Ausubel,  1980).  The  content  was  chosen 
for  its  appropriateness  to  the  overall  course  content  and  for  its  presentation  of 
coordinate  concepts. 

The  materials  for  the  treat^'^^nt  group  also  contained  a  graphic  advance 
organizer  in  the  form  of  a  concept  tree,  preceding  tiie  text  passage*  The  concept  tree 
illustrated  each  of  tiie  key  concepts  presented  in  tiie  text  passage,  tiie  hierarchical 
relationship  between  the  concepts,  and  an  example  of  each  concept.  No 
information  was  added  to  the  concept  tree  that  did  not  appear  in  the  text  passage. 
Both  tiie  format  and  the  content  of  tiie  concept  tree  were  validated  tiirough 
consultation  witii  a  subject  matter  expert  in  the  areas  of  learning  theory  and 
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instructional  design.  The  presence  or  absence  of  the  concept  tree  constituted  the 
independent  variable  for  this  study. 


Figure  1  about  here 


The  materials  also  included  a  cover  page,  explaining  that  a  new  unit  was  being 
considered  for  inclusion  in  tiie  course,  and  the  attached  material  represented  a  'try- 
out*  of  tiiat  unit.  It  further  stated  that  students  would  have  20  minutes  to  read  tiie 
material,  after  which  a  short  questionnaire  and  quiz  would  be  administered. 

The  cover  page  for  tiie  experimental  group  contained  an  additional 
paragraph  describing  the  function  of  the  concept  tree  and  directions  for  use.  The 
paragraph  also  infomied  students  to  study  tiie  concept  tree  prior  xo  reading  and 
encouraged  students  to  refer  to  tiie  concept  tree  during  rea(ing. 

Dependent  Measures 

Two  instruments  were  used  to  measure  tiie  effects  of  six  dependent 
variables.  The  instruments  included  an  immediate  posttest  and  the  Instructional 
Materials  Motivation  Scale  (IMMS). 

Immediate  posttest  -  An  1 1  item  posttest  served  as  a  dependent  mea  jure  of 
immediate  concept  acquisition.  The  items  consisted  of  six  fill-in-the-blank  and  five 
multiple  choice  questions.  The  questions  asked  students  to  recall:  (1)  the  definition 
of  each  concept,  (2)  tiie  hierarchical  relationship  between  concepts,  and  (3)  tiie 
name  of  the  concept  associated  witii  presented  examples.  A  subject  matter  expert  in 
educational  psychology  validated  the  foraiat  and  content  of  the  posttest  To  gau^^e 
the  internal  consistency  of  tiie  posttest,  a  split-half  reliability  coefficient  was 
calculated  for  posttest  scores.  As  corrected  with  the  Spearman-Brown  prophecy 
formula,  tiie  reliability  coefficient  was  .71. 

Instructional  Materials  Motivation  Scale  flMMS)  -  A  modified  version  of  tiie 
IMMS,  developed  by  Keller  (1988),  was  used  to  measure  tiie  motivational  impact 
of  tiie  instructional  materials.  Based  on  tlie  ARCS  model  of  motivational  design, 
tiie  original  version  of  tiie  IMMS  contained  36  items  related  to  the  4  primary 
dimensions  of  motivation  proposed  by  Keller  (1987)-Attention,  Relevance, 
Confidence,  and  Satisfaction.  Due  to  the  time  limitations  of  the  course,  6  items 
from  each  of  tiie  4  dimensions  were  selected  for  this  study.  Each  item  required 
subjects  to  indicate  strength  of  agreement  with  statements  regarding  motivational 
aspects  of  tiie  instructional  materials  on  a  4  element  Likert  scale.  The  effect  of  the 
concept  tree  on  overall  motivation  was  calculated  by  summing  the  scores  of  each  of 
tiie  four  subscales.  The  subscales  were  also  examined  independendy  to  measure 
tiie  effects  of  the  concept  tree  on  attention,  relevance,  confidence,  and  satisfaction. 
The  standardized  item  alpha  coefficient  of  reliability  for  tiie  overall  IMMS  was 
calculated  as  .90.  Subscale  reliability  coefficients  were:  (1)  Attention  =  .74;  (2) 
Relevance  =  .67;  (3)  Confidence  =  .68,  and;  (4)  Satisfaction  =  .82. 

Procedures 

On  tiie  day  of  the  experiment,  each  student  was  given  a  copy  of  tiie  assigned 
learning  material,  based  on  tiiat  student's  assignment  to  treatment  or  control.  All 
students  were  then  directed  to  read  tiie  cover  page  on  tiie  learning  materials,  as  tiie 
instructor  read  it  aloud  The  directions  informed  the  smdents  that  the  materials 
represented  a  trial  of  a  new  learning  unit  being  considered  for  subsequent  offerings 
of  tiie  course,  and  that  students  would  be  given  20  minutes  to  read  the  materials. 
The  instructor  then  directed  students  to  turn  tiie  page  and  begin. 

Smdents  were  allowed  20  minutes  to  read  tiie  instructional  text,  after  which 
they  were  asked  to  complete  the  motivation  scale.  Following  tiie  completion  of  the 
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scale,  the  instructional  materials  were  collected  and  a  posttest  was  administered. 
Students  were  given  15  minutes  to  complete  the  test 

Statistical  Analysis 

A  2  X  3  factorial  design  was  employed  for  this  study,  in  which  tlie  variables 
were  treatment  (concept  tree  vs.  no  concept  tree)  and  reading  ability  (low  vs. 
average  vs.  high).  All  statistical  analysis  in  this  investigation  were  perfomKJd  with 
the  commercially  available  SPSS  program  (Hull  &  Nie,  1981). 

For  each  dependent  measure,  a  visual  inspection  of  score  distributions  was 
performed  Each  set  of  data  appeared  consistent  with  an  assumption  of  normality. 
Following  this,  homogeneity  of  variance  was  tested  for  each  of  the  measures,  using 
the  Fmax  test  (.05  level).  Because  cell  sizes  differed,  a  conservative  approach  to 
determinmg  degrees  of  freedom  was  taken  (i.e.,  degrees  of  freedcMn  for  the  smallest 
cell  was  used  for  purposes  of  comparison).  In  all  cuses,  variance  ratios  were  below 
the  critical  value  at  p=.05,  Fmax  (6,9)  =  7.80,  and  homogeneity  was  assumed  A 
two-factor  analysis  of  variance  (treatment  x  ability  level)  was  conducted  for  each 
dependent  measure. 

Results 

Data  comparing  the  effects  of  the  concept  tree  on  coordinate  concept 
acquisition  and  learner  motivation  is  presented  in  table  1.  Since  no  interactions 
were  revealed  in  the  analyses,  a  simply  summary  of  mean  scores  for  treatment  and 
control  groups  on  all  dependent  measures  are  presented. 


Table  1  about  here 


Posttest  Achievement 

The  two-factor  (treatment  x  ability  level)  analysis  of  variance  revealed  a 
significant  main  effect  of  treatment,  F(l,67)=7.98,  p<.05.  This  indicates  that  the 
average  post  test  achievement  score  of  subjects  receiving  the  concept  tree  was 
reliably  higher  than  that  of  subjects  who  did  not  receive  the  concept  tree.  No 
significant  main  effect  of  ability  level  was  found,  F(2,67)=1.46,  p>.05.  The 
interaction  was  also  non-significant,  F(2,67)=.85,p>.05.  The  significant  main 
effect  of  treatment  supports  tiie  first  research  hypothesis.  Tlie  lack  of  significance 
for  the  aptitude  treatment  interaction  suggests  that  the  answer  is  **no"  to  the  research 
question,  "Do  students  with  different  reading  abilities  experience  differential  gains 
in  achievement  when  presented  with  a  concept  tree  as  a  graphic  advance  organizer?" 

A  two-factor  ANOVA  similar  to  that  conducted  on  posttest  scores  was 
conducted  on  overall  scores  for  tiie  IMMS.  As  with  the  posttest,  a  significant  main 
effect  of  treatment  was  discovered,  F(l,67)=8.587.  p<.05.  This  supports 
hypothesis  #2,  indicating  tliat  the  presentation  of  a  concept  tree  may  increase 
students*  perceiveu  iCvel  of  overall  motivation.  Neither  the  main  effect  of  ability 
level  nor  the  interaction  were  found  to  be  significant 

Virtually  identical  results  were  found  in  3  of  the  4  ANOVAS  performed  on  the 
IMMS  subscale  scores.  The  main  effect  of  treatment  was  found  to  be  significant  in 
the  Attention  subscale,  F(l,67)=7.59,  p<.05,  tiie  Confidence  subscale, 
F(l,67)=8.30,  p<.05,  and  tiie  Satisfaction  subscale,  F(l,67)=8.05,  p<.05.  No 
statistically  significant  interactions  between  ability  ^;id  treatment  were  found  for  any 
of  the  four  subscales.  No  significant  main  effects  or  interactions  were  revealed  on 
the  Relevance  subscale.  The  significance  of  the  main  effect  of  treatment  in  the 
Confidence  subscale  supports  research  h)TX)thesis  #3;  students  receiving  the 
concept  tree  as  a  graphic  advance  organizer  for  tiie  text  passage  reported 
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significandy  higher  levels  of  perceived  confidence  than  subjects  not  receiving  the 
concept  tree. 

Discussion 

The  ability  to  efficiently  identify  and  acquire  cocoxiinate  concepts  presented 
in  text  passages  is  believed  to  be  crucial  to  the  academic  success  of  university 
students.  Thus,  in  order  to  gain  a  better  understanding  of  reading  compirhension, 
it  is  important  to  determine  both  the  cognitive  and  motivational  effects  of 
instructional  tools  designed  to  facilitate  reading.  This  study  examined  the  effects  of 
providing  a  concept  tree  along  with  text  information  on  coordinate  concept 
acquisition  and  learner  motivation. 

The  results  indicate  that  the  presentation  of  a  concept  tree  may  increase 
students'  ability  to  recall:  (1)  the  definition  of  coordinate  concepts,  (2)  the 
hierarchical  relationship  between  concepts,  and  (3)  the  name  of  the  concept 
associated  with  presented  examples.  It  seems  apparent  that  the  addition  of  a 
diagram  illustrating  the  relationship  of  key  concepts  would  help  students  identify 
and  epcode  coordinate  concepts.  The  focus  of  this  study,  however,  is  to 
demonstrate  that  a  relatively  simple  procedure  (i.e.,  developing  a  concept  tree) 
facilitates  studenb*  ability  to  identify  and  acquire  cc*.^axiinate  ccmcepts,  thereby 
giving  both  students  and  instructors  more  time  to  concentrate  on  hi^er  order 
thinking  skills.  The  results  also  lend  additional  evidence  to  the  premise  that  the  use 
of  graphic  advance  organizers  can  improve  learning  from  text  (Moore  and 
Readence,  1984). 

The  presentation  of  a  concept  tree  is  also  associated  with  increased  levels  of 
perceived  motivation.  Students  receiving  the  concept  tree  repoited  sigpificanUy 
higher  levels  of  overall  motivation,  attention,  confidence,  and  satisfaction*  This 
suggests  tiiat  college  smdents  receiving  a  concept  tree  may  extend  m<KC  effort  to 
learn  information  delivered  tiirough  text-based  instruction.  More  specifically,  it 
appears  tiiat  a  concept  tree  may  increase  perceptual  arousal  and  enhance  students' 
perceived  abiUty  to  read  difficult  text  passages  with  meaning.  In  turn,  smdents  who 
gain  substantial  amounts  of  knowledge  from  reading  assignments  while  expending 
a  reasonable  amount  of  effort  may  experience  higher  levels  of  perceived  satisfaction 
compared  to  students  having  difficulties  in  reading.  The  concept  tree,  however,  did 
not  appear  to  affect  perceived  levels  of  relevance.  Smdents  receiving  the  concept 
tree  may  have  experienced  equal  levels  of  perceived  relevance  compared  to  smdents 
not  receiving  die  concept  tree  because  relevance  may  be  more  of  a  fiinction  of 
content  tiien  presentational  style. 

In  light  of  studies  tiiat  have  demonstrated  the  differential  effects  of 
instruction  on  cognition  and  motivation  (Alderman  &  Mahler,  1973;  Johnston, 
1975),  tiie  importance  of  examining  both  factors  during  the  design  and 
implementation  of  instruction  is  unequivocal.  This  smdy  indicated  that  the  use  of 
concept  trees  as  advance  organizers  may  increase  student  motivation  and  tiieir 
ability  to  acquire  coordinate  concepts.  Thus,  concept  trees  may  facilitate  learning 
and  enhance  students*  desue  to  initiate,  dinect,  and/or  persist  at  text-based 
instruction. 

One  major  caveat  must  be  stated  Regardless  of  tiie  presence  of  tiie  concept 
tree,  posttest  performance  scores  were  relatively  low.  Altiiough  significant,  the 
difference  in  average  posttest  scores  of  subjects  who  had  accers  to  a  GAO  rjid 
those  who  did  not  represents  slightly  more  than  one  additional  question  correct.  In 
conjunction  with  large  standard  deviations  representing  somewhat  flat  distributions 
of  suident  scores,  tiie  reader  is  cautioned  that  issues  of  effect  size  must 
considered  when  interpreting  the  results  of  the  achievement  test 
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Alternatively,  a  different  explanation  for  low  posttest  performance  can  be 
offered.  The  instructional  text  used  for  this  study,  while  relatively  short,  was 
difficult.  When  coupled  with  the  short  duration  of  reading  time,  low  performance 
scores  arc  not  unexpected.  Students  arc  frequently  required  to  deal  wit*'  text  that  is 
difficult  or  potentially  confusing,  especially  in  an  unfanuliar  area  of  study.  Viewed 
in  this  light,  the  improved  posttest  performance  in  this  study  of  subjects  who  had 
access  to  a  GAO  must  not  be  overlooked 

Perfiaps  the  most  significant  new  dilution  identified  in  this  research  is  the 
effect  of  graphic  organizers  on  subjects'  motivational  states.  To  date,  few  studies 
have  examined  the  simultaneous  impact  of  graphic  organizers  on  motivation  and 
cognition.  Typically,  prior  studies  which  have  dealt  with  motivational  aspects  of 
graphic  organizers  have  examined  the  naotivation  of  students  to  use  the  organizers 
during  instruction  (Moore  and  Readence,  1984).  This  has  been  viewed  largely  as  a 
function  of  the  perceived  "fit"  between  the  organizer  and  the  material  it  was 
designed  to  support.  As  mentioned  earlier,  the  congruence  between  GAO  and 
instructional  text  in  this  research  was  validated  through  consultation  with  a  subject 
matter  expert;  hence,  a  good  match  was  presumed 

Given  the  correct  "fit"  described  above,  and  the  demonstrated  increases  In 
motivation  associated  with  the  graphic  organizer,  several  questions  rcmaiii  to  be 
investigated.  First,  what  are  the  specific  features  of  the  organizer,  separately  and  in 
conjunction  with  the  text  it  is  designed  to  support,  that  arc  related  to  increases  in 
motivation?  An  analysis  and  quantification  of  such  features  niay  lead  to  a 
systematic  and  replicable  method  for  producing  graphic  organizers  which,  in  turn, 
would  be  related  to  consistent  increases  in  student  motivation  over  text  which 
contains  no  such  organizer.  A  second,  and  perhaps  more  important,  question 
pertains  to  the  relationship  between  motivation  and  performance.  Presuming  that  a 
system  could  be  devised  for  producing  graphic  organizers  that  consistently  improve 
students*  motivation  toward  instructional  materials,  what  would  be  the  subsequent 
effect  on  performance  in  terms  of  retention,  transfer,  and  cognitive  style?  WiU 
students  presented  regularly  with  a  conce])t  tree  begin  to  use  this  st^tegy  in  other 
instructional  situations  such  as  reviewing  for  a  test?  This  question  can  only  be 
answered  after  a  system  ior  producing  consistent,  replicable,  and  high-quality 
organizers  has  been  developed  Frrilly,  this  study  examined  gains  associated  v/ith 
a  single,  isolated  exposure  to  a  GAO.  Furtlier  iiesearch  is  required  on  the 
cumulative  rnipact  of  such  devices  as  an  ongoing  or  regular  instructional  stratef<y. 
In  addition,  research  that  examines  the  effectiveness  of  GAOs  on  passages  ot 
varying  difficulty  levels  and  on  passages  that  are  aimed  at  different  t>pes  of  learning 
(e.g.,  verbal  information,  rules,  or  cognitive  strategies)  will  provide  a  more 
balanced  picture  Oi  the  overall  f*.fficacy  of  tiiese  devices* 

The  results  of  this  research  appear  to  support  the  efficacy  of  graphic 
organizers  in  terms  of  both  concept  acquisition  and  i£X)tivation.  Given  ihe  ease  with 
which  such  organizers  can  be  crcated,  and  the  relatively  low  cost  in  temis  of  time 
and  required  materials,  they  reprcsent  a  potentially  valuable  tool  for  facilitating  the 
acquisition  of  coordinate  concept  information.  In  addition,  Uiis  smdy  presents  a 
pragmatic  method  of  examining  both  the  cognitive  and  motivational  effects  of 
instruction.  In  order  to  gain  a  better  undei^standing  of  tki  factors  which  affect 
human  performance,  we  need  to  take  into  account  the  imj  act  of  iniJtmction  on 
cognition  and  motivation. 
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Table  1 .  Mean  scor<^>>  for  depende>nt  measures 


Concept  tree 

No  concept  tree 

Dependent  measure 

M 

SD 

M 

SD 

Posttest* 

46.90 

18.97 

34.97 

17.29 

IMMS  (24  items)** 

49.55 

12.18 

41.71 

10.29 

Attention  (6  items)** 

11.68 

3.91 

9.49 

2.74 

Relevance  (6  items)** 

14.92 

3.45 

13.77 

3.29 

Confidence  (6  items)** 

12.32 

3.74 

9.94 

3.40 

Satisfaction  (6  items)** 

10.66 

3.63 

8.49 

2.64 

*  Percentage  score 

**  Sum  of  points  for  ail  items,  each  item  wiih  point  range  of  1-4 
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CATEGORIES  OF 
MEANINGFUL  LEARNING 


Concept  Learning 
Focmuioo  &  Afsimilation 


Propositional 
Learning 


o 


ExompUs 


Firsr  grajers  leain  the 
word  "dinosaur"  after 
viewing  some  pictures 
and  models  of  various 
examples. 


After  studying  the 
concept  of  sailboats, 
students  are  shown 
exiunples  of  many 
different  types  and 
classes  such  as;  lasers, 
sunfishes,  and  cata- 
marans. 


After  being  introduced 
to  the  concept  of 
shore  birds,  students 
learn  that  the  birds 
are:  1)  found 
coasthnes,  and 
2)  typically  eat  fish 
and  other  marine  life. 


After  listening  to  and 
discussing  tfa^  music 
of  several  composas, 
music  students  learn 
that  all  of  the  com- 
posers wrote  during 
the  Renaissance 
period. 


During  physical  educa- 
tion«  students  first 
leam  how  to  play  bas- 
ketball. Then  the 
students  leani  how  to 
play  baseball. 


Figure  1.  The  concept  tree  illustrates  the  relationships  between  key  concepts  discussed  by  Ausubel. 
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FACULTY  INNOVATORS  AT  A  LARGE  UNIVERSITY 
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INTRODUCTION 

In  settings  of  higher  education,  specially,  large  universities, 
nio&t  instructional  innovations  are  designed,  developed,  and  used 
by  faculty  members  themselves.  The  Educationsil  Development 
Program  (EDP),  Michigan  State  University  relied  on  such  voluntary 
vacuity  initiatives  to  bring  about  instructional  innovations  in 
various  departments.  In  this  kind  of  environment  individual 
faculty  member  feels  the  need  for  change,  decides  to  do  something 
about  it,  and  takes  needed  steps  to  introduce  new  ideas  or 
objects  to  affect  the  desired  change.  This  voluntary  approach, 
therefore,  is  considerably  different  from  the  more  traditional 
approach  in  which  a  change  agent  from  outside  the  system 
identifies  the  need  for  change  and  tries  to  motivate  the  members 
of  an  organization  to  adopt  a  particular  innovation  to  meet  thai 
need. 

Faculty  members  who  feel  the  need  for  improving  their 
instruct  ional  processes,  and  actual ly  take  act  ions  to  sat  isfy 
such  needs,  are,  in  the  words  of  Rogers  and  Shoemaker,  called 
innovators?  and  in  the  words  of  Steiner,  creative  individuals. 
Innovators  or  creative  Individuals  are  a  highly  motivated  group 
compared  to  othier  categories  of  adopters. 

Innovators  are  considered  to  be  advent u*^esome,  starry-eyed,  or 
experimenters.  They  deal  with  ideas  and  activities  that  are 
avant-garde,  hazardous,  rash  or  risky.  They  are  usually  able  to 
understand  complex  technical  ideas  and  products,  and  are  not 
disturbed  by  repeated  failures.  They  are  usually  young,  have 
high  social  status  (including  education,  prestige  and  income), 
rely  on  impersonal  and  cosmopolitan  sources  of  information,  exert 
opinion  leadership,  and  are  regarded  by  their  peers  as  being 
deviant  and  unusual  individuals  (Rogers,  1965).  Steiner' s 
summary  of  findings  (1965)  dealing  with  creative  individuals  also 
highlight  similar  attributes  of  such  persons.  Besides  the 
characteristics  or  attributes  that  usually  mark  the  creative 
individuals  as  deviant,  Steiner  also  ifidentifies  the  following 
attributes:  conceptual  fluency,  ability  to  produce  a  large  number 
of  ideas  quickly,  originality  in  generating  unusual  ideas, 
ability  to  separate  source  from  content  in  evaluating 
mf ormat  ion,  mot  i vated  by  a  deep  interest  in  the  problem  faced 
and  willing  to  follow  the  problem  wherever  it  leads.  Over  and 
above,  he  suspends  judgment  and  avoids  early  commitments,  he  is 
less  authoritarian,  he  accepts  his  own  impulses  and  is  playful 
and  undisciplined  in  his  explorations,  he  exercises  independence 
of  judgment  and  is  not  prone  to  conformity;  and  while  he  has  a 
rich  and  even  ''bizarre"  fantasy  life,  he  has  a  superior  reality 
or ientat ion. 
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It  IS,  therefore,  quite  obvious  that  such  individuals  will  face 
great  difficulty  in  conforming  to  the  routine  demands  of 
educational  settings.  Davi et.  al.,  (1976)  found  that  a  number 
of  instructional  innovators  at  Michigan  State  University 
consciously  violated  the  role  expectations  of  their  colleagues 
and  played  the  part  of  "dissatisfied  mavericks". 

These  highly  motivated  individuals,  however,  do  not  innovate  and 
adopt  new  ideas  or  products  without  being  affected  by  the 
innovation  and  the  organization  or  system  in  which  they  operate. 
A  number  of  studies  have  focused  on  different  aspects  of 
individual,  innovation  and  organizational  variables  which  play 
significant  roles  in  innovation-adoption  processes.  Researchers 
(Miles,  1364?  Diamond  et.  al.,  1975  and  Davis,  1979)  have  also 
emphasized  the  importance  of  departmental  support  for  innovations 
and  continued  use  of  these  innovations. 

This  study,  therefore,  has  attempted  to  find  out  whether 
departmental  support  plays  a  significant  role  instructional 
innovations,  created  and  adopted  by  faculty  innovators  in  large 
universities.  It  should  be  noted  that  in  large  universities 
faculty  members  have  relatively  more  autonomy  to  decide  on  the 
content  of  their  courses,  and  how  they  should  be  taught,  so  long 
as  the  courses  conform  to  the  descriptions,  and  the  resources  are 
at  hand    (Davis,    et.    al.,  1980). 


RESEARCH  QUESTIONS 

In  order  to  investigate  this  problem,  the  study  has  attempted  to 
answer  the  following  research  questions: 

(1)  Is  there  any  significant  relationship  between  general 
departmental  support  and  adoption  (building  and  use)  of 
instructional  innovations  by  faculty  innovators  at  a 
large  university? 

(£)  Is  there  any  significant  relationship  between  specific 
departmental  supports  and  adoption  of  instructional 
innovations  by  faculty  innovators? 

For  greater  understanding  basic  terms  and  expressions  used  in 
this  study  are  defined  and  described  below: 

General  Departmental  Support  refers  to  resources  of  an  academic 
department  that  can  be  used  in  building  and  use  of  instructional 
innovat  ions. 

Specific  Departmental  Supports  are  financial,  policy,  technical, 
office,    colleagues'    and  chairman's  support. 

Instructional  Innovations  refer  to  projects  designed,  developed 
and  used  for  the  purpose  of  improving  teaching- learning  processes 
in     different       departments       at       Michigan     .State  University. 
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Faculty  Innovators  are  those  faculty  rnembRrs  who  took  voluntary 
initiatives  to  develop  and  use  EDP  instructional  projects  to 
improve  their  instructional  processes  in  different  departments 
of  Michigan  State  University. 

Adoption  refers  to  the  actual  building  and  use  of  instructional 
innovations  by  faculty  innovators* 


METHODOLOGY 


POPULATION  AND  SAMPLE 


The  study  employed  a  cross-sectional  survey  method-  The 
population  for  the  study  consisted  of  all  faculty  members  (also 
called  project  directors),  who  developed  and  used,  on  a  voluntary 
basis,  instructional  innovations  in  their  departments  at  Michigan 
State  University.  In  order  to  include  only  one  director  from 
each  project,    the  following  criteria  were  followed: 

(^)  The  project  director  must  be  a  faculty  member  in  an 
academic  department  at  Michigan  State  University  with 
teaching  as  the  prima*^y  responsibility. 


(^)  Efforts  were  made  to  include  the  major  project  director 
from  each  project  in  the  sampling  frame.  If  the  major 
project  director  was  not  available,  then  the  next  faculty 
project  director  was   included,    and  so  on. 


During  the  period  under  study,  a  total  of  180  instructional 
projects  were  repo>'^ted.  Out  of  these  180  projects,  50  were 
excluded  (because  they  were  either  continuations  of  previous 
projects,  or  they  had  directors  involved  in  more  than  one  project 
at  a  time),  before  a  sample  of  65  projects  was  drawn  by  using 
random  sampling  technique. 


DATA  COLLECTION 


To  collect  the  data  from  the  respondents  a  structured 
questionnaire  with  a  S-point  Likert  scale  was  designed,  developed 
and  validated.  A  total  of  thirty  statements  were  used  on  the 
questionnaire  to  collect  respondents'  perceptions  with  reqard  to 
departmental  support  and  adoption  of  instructional  innovations. 
The  respondents  were  asked  to  indicate  the  degree  of  their 
agreement  or  disagreement  with  the  questionnaire  statements. 
Numerical  weights  were  assigned  to  each  of  5-response  categories 
in  a  way  that  the  higher  the  score,  the  greater  the  degree  of 
agreement  or  disagreement. 

The  questionnaires  were  mailed  to  the  respondents  with  cover 
letters  assuring  the  maintenance  of  confidentiality  and 
anonymity.    A  three-week  period  was  allowed  for  the  return  of  the 
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completed  questionnaires.  Through  telephone  calls,  one  more 
attempt  was  made  to  collect  data  from  those  who  failed  to  return 
the  completed  questionnaires  in  the  first  place.  P  total  of  55 
completed  questionnaires  were  received;  the  rate  of  return  being 
84  percent . 


DATA  ANALYSIS 

The     data     were     analyzed     by     the     computer     using     SPSSX.  The 
statistical   techniques  used  were  Pearson  correlation  and  partial 
correlation.        To     find     out     the     relationship     between  general 
departmental   support  and  adoption  PeariBon  correlation     was  used, 
while  partial   correlation  analysis  was  employed  to  det©rmi»ie  the 
relationship  of  each  specific  departmental  support  with  adoption 
controlling  other  types  of  departmental     support.        For  example, 
the  relationship     between     financial     support     and     adoption  was 
measured  by     controlling     other     independent     variables,  namely, 
policy,    technical,    office,    colleagues'      and     chairman's  support, 
and  so  on. 


FINDINGS 


The  findings  of  the  study  indicate  a  significant  relationship 
(r  =  .4609,  p  <  0.05)  between  general  departmental  support  and 
adoption  of     instructional      innovations     by     faculty  innovators. 

With  regard  to  the  relationship  between  specfic  departmental 
supports  and  adoption,  the  findings  revealed  significant 
relationships  between  office  support  and  adoption,  colleagues' 
support  and  adoption,  and  chairman's  support  and  adoption.  No 
->ignif  leant  relationships  were  reported  between  financial 
jbLipport  and  adoption,  policy  support  and  adoption,  and  technical 
support     and     adoption.       The     table     below     contains     the  data. 


Relationship  between  specific  departmental 
supports  and  adopt  ion 
(n=:55) 


Speci f ic  support 

Partial  r 

P 

Financial  support 

.  0173 

•  452 

Policy  support 

£812 

.  024 

Technical  support 

1542 

.  143 

Office  support 

. 4905* 

.  000 

Col  leagues'  support 

. 3247* 

.  011 

Chairman' s  support 

. £865* 

.  022 

*  Significant  at   .05  level. 
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R  Significant  relationship  between  general  departmental  support 
and  adoption  suggests  that  the  respondents  perceived  a  positive 
role  of  departmental  support  in  relation  to  adoption  of 
innovations.  Departmental  support  or  resources,  therefore,  can 
be  profitably  used  to  manipulate  the  innovations-adoption 
behaviors     of       faculty       innovators       in       large  universities. 

To  further  examine  the  relationship  between  independent  and 
dependent  variables,  departmental  support  was  divided  into  six 
types  of  support,  namely,  financial,  policy,  technical,  office, 
colleagues'  and  chairman's  support.  The  respondent©  perceived 
office,  colleagues'  and  chairman's  support  as  important  factors 
in  relation  to  their  innovat ion-adopt ion  behaviors.  It  is 
interesting  to  riote  that  the  respondents  did  not  perceive  a 
significant  relationship  between  financial  support  and  their 
adoption  behaviors.  Some  of  the  possible  reasons  for  this,  may 
be,  due  to  the  amount  of  funds  needed,  not  easily  available 
funds,  and  the  personality  characteristics  of  the  respondents.  If 
an  innovation  requires  a  small  dollar  investment,  it  is  likely 
that  the  adopter,  who  is  highly  motivated  to  bring  about  a 
change,  may  net  perceive  the  need  for  such  funds  as  being 
important.  Policy  support  was  also  not  perceived  significant  by 
the  respondents.  Considering  the  personality  traits  of 
i.'tnovators  and  the  extent  of  autonomy  faculty  members  enjoy  in 
respect  of  their  teaching  in  large  universities,  it  can  be  said 
that  they  will  not  consider  policy  support  as  an  important  factor 
in  relation  to  their  adoption  bahaviors  as  long  as  instructional 
innovations  do  not  affect  others  in  the  organization  or  the 
organisation  itself.  Since  innovators  are  experimenters, 
risk— takers  and  quick  to  grasp  complex  and  technical  ideas  and 
products,  it  IS  no  wonder  why  they  will  perceive  technical 
support  as  an  important  factor  influencing  their  willingness  to 
design,    develop  and  use   instructional  innovations. 
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Layout  Decision  Mal(ing: 
The  Placement  of  Illustrations 

Introduction 

A  necessary  process  for  the  incorporation  of 
illustrations  in  instructional  text  is  to  determine  the  way  in 
which  the  illustration  joins  with  the  text  to  create  a  pleasing 
and  effective  page  design.    The  effects  of  illustrations  in 
instructional  texts  has  received  considerable  research 
attention.    The  research  suggests  that  illustrations  not  only 
provide  interest,  motivation,  and  attractiveness,  but  also  aid 
the  recall  and  retention  of  information  (Levie  and  Lentz,  1982; 
Anglin,  1987). 

The  advent  of  desktop  publishing  has  affected 
instructional  text  design.    It  is  now  easy  for  designers  to 
manipulate  illustrations  and  text;  illustrations  can  be 
positioned  anywhere  on  a  page  with  the  text  "flowing  around 
them."  Consequently,  many  page  designs  confuse  the 
sequential  organization  of  the  text.   What  is  thought  to  be  an 
obvious  organizational  text  pattern,  may  require  learners  to 
examine  words  in  an  attempt  to  determine  where  they  are  to 
read  next.  For  example,  what  happens  when  the  normal 
horizontal  or  vertical  flow  is  disrupted  by  a  picture  or 
graphic?    This  type  of  positioning  of  illustrations  in  relation 
to  the  text  has  received  little  research  attention  (Hartley, 
'987).  Yet  is  seems  possible  that  reading  speed  is  slowed  and 
t  ie  thought  process  disrupted  if  the  reader  must  stop  and 
jearch  for  the  remainder  of  a  sentence  or  the  next  paragraph. 

Designers  need  to  consider  how  the  illustrations  join 
with  the  text  so  that  the  overall  page  is  pleasing  and  effective 
for  the  learner.   The  design  elements  should  be  manipulated  so 
that  information  in  the  text  is  presented  in  an  organized  and 
consistent  manner.    For  the  most  part,  written  text  is 
sequenced  in  a  left-to-right,  top-down  direction.  Therefore, 
horizontal  arrangements  will  tend  to  be  perceived  in  a  left-to- 
right  sequence,  and  vertical  arrangements  in  a  top-down 
sequence  (Hartley,  1987). 

This  study  investigates  the  positioning  of  illustrations 
and  how  they  affect  the  sequential  organization  of  the  text. 
The  primary  purpose  of  the  study  is  to  describe  learners' 
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reading  sequence  and  preference  for  page  designs  with 
variations  of  illustration  placement. 

Method 

Subjects  and  Design 

The  70  subjects  were  undergraduate  and  graduate 
students  enrolled  in  education  courses  at  Memphis  State 
University.    A  within-subjects  design  was  used,  with  subjects 
responding  to  each  page  design. 

Materials 

The  eight  one-page  "articles"  used  in  the  study  were 
designed  with  desktop  publishing  software  and  printed  with  a 
laser  printer  to  approximate  a  published  page.   For  the 
preferred  or  "natural"  reading  sequence  of  the  subjects  to  be 
determined,  meaningless  text  which  had  the  same  structure 
and  appearance  as  realistic  text  (Shannon  and  Weaver,  1964; 
Ross  and  Morrison,  1989)  was  used.  The  text  was  printed  in  a 
two-column  format  for  two  of  the  designs  and  in  a  three- 
column  format  for  the  other  six.   In  each  design,  the 
illustration  "cuts"  across  at  least  two  columns  (see  Figure  1). 
Another  variable,  a  two-point  horizontal  bar,  was  inserted 
above  the  illustration  in  four  of  the  designs.    Each  individual 
page  design  is  described  below. 

1 .  Page  design  #1 :  Two  column  format  with  the 
illustration  centered;   the  illustration  breaks  both 
columns,  dividing  the  page  in  half. 

2.  Page  design  #2:  Identical  to  design  #1  with  a  2-point 
horizontal  rule  inserted  above  the  illustration. 

3.  Page  design  #3:  Three  column  format  with  the 
illustration  centered;   the  illustration  breal''^  all  three 
columns,  dividing  the  page  in  half. 

4.  Page  design  #4:  Identical  to  design  #3  with  a  2-point 
horizontal  rule  inserted  above  the  illusLation. 

5.  Page  design  #5:  Three  column  format  with  the 
illustration  centered;   the  illustration  breaks  the  left 
and  middle  columns. 

6.  Page  design  #6:  Identical  to  design  #5  with  a  2-point 
horizontal  rule  inserted  above  the  illustration. 
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7.  Page  design  #7:  Three  column  format  with  the 
illustration  centered;   the  illustration  breaks  the  middle 
and  richt  columns. 

8.  Page  design  #8:  Identical  to  design  #7  with  a  2-point 
horizontal  rule  inserted  above  the  illustration. 

A  10-item  survey  was  also  developed.  Demographic 
items  consisted  of  statements  concerning  student 
classification,  sex,  GPA,  age  and  major.   Additional  items 
consisted  of  statements  about  reading  speed,  comprehension, 
subject's  primary  reading  source,  and  whether  or  not  the 
subject  likes  to  read. 

Procedure 

Subjects  were  tested  in  a  group.  Each  subject  was  given 
a  set  of  stapled  pages  containing  the  randomly  sequenced  page 
designs.  Each  design  was  separated  by  a  response  sheet.  For 
each  design  subjects  were  asked  to  trace  the  sequence  with 
which  they  would  naturally  read  the  text  with  a  pencil.  After 
tracing  their  preferred  reading  format,  subjects  indicated 
their  preference  for  the  design  on  a  5-point  Likert  scale  with 
5  representing  the  most  positive  reaction.    Subjects  were 
instructed  not  to  look  back  at  any  previous  design.  After 
responding  to  all  eight  designs,  the  subjects  completed  the 
10-item  survey. 

Results 

The  analysis  of  subjects'  reading  paths  identified  three 
patterns  of  reading  sequence.    The  most  frequently  identified 
pattern  was  a  top-down,  left-to-right  reading  sequence  (see 
Table  1).   The  pictures  made  no  significant  difference  to  these 
subjects.    Regardless  of  picture  placement,  the  subjects 
continued  to  read  the  columned  text  in  a  top-down  fashion. 
The  second  most  common  pattern  was  a  left-to-right,  top- 
down  sequence.   The  illustration  served  as  a  text  boundary 
causing  the  subjects  to  read  the  blocks  of  text  above  the 
illustration  and  then  the  text  below  the  illustration.  Another 
sequence  which  was  identified  by  a  small  percentage  of 
subjects  with  page  design  #6  was  a  left-to-right,  top-down, 
left-to-right  pattern.    The  subjects  read  around  the 
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illustration  as  if  the  material  under  it  had  been  blocked  off  as 
descriptive  information. 

Most  subjects  utilized  a  consistent  path  sequence 
regardless  of  picture  placement  or  rule  insertion.  However, 
the  percentages  indicate  that  the  placement  of  rules  did  guide 
some  of  the  subjects'  reading  patterns.   Many  subjects  showed 
a  different  reading  pattern  for  page  designs  which  were 
identical  except  for  the  insertion  of  a  rule.  Twenty-one 
percent  of  the  subjects  who  responded  to  page  design  #1  with 
a  top-down,  left-to-right  reading  sequence  changed  their 
reading  sequence  with  design  #2  to  a  left-to-right,  top-down 
sequence.     Likewise,  26  percent  responded  differently  to 
designs  #3  and  #4;  18  percent  to  designs  #5  and  #6;  and  14 
percent  to  designs  #7  and  #8. 

The  analysis  of  subjects'  preference  for  page  designs 
indicate  that  a  preference  for  two  column  material  over  three 
column  material  exists.   The  least  preferred  page  designs  were 
#3  and  #4  where  the  illustration  cut  across  all  three  columns. 

Discusfiion 

The  results  of  this  study  imply  that  illustration 
placement  is  a  critical  attribute  to  be  considered  in  the  design 
of  instructional  text.    The  study  suggests  that  a  two-column 
format  combined  with  illustrations  dividing  all  columns  is 
preferred  over  a  three-column  format  incorporating  an 
illustration  dividing  all  columns,    It  would  appear  that  larger 
blocks  of  text  support  the  top-down,  left-to-right  reading 
pattern,  and  are  more  pleasing  to  the  reader.   However,  more 
than  half  of  the  subjects  were  neutral  or  negative  about  all 
eight  page  designs.   This  result  suggests  that  readers  do  not 
particularly  like  designs  where  illustrations  are  positioned  in 
the  middle  of  the  text.    Designs  where  illustrations  are 
positioned  above  or  below  the  te:<t  were  not  incorporated  into 
this  study  because  these  combinations  do  not  affect  the 
sequential  organization  of  the  text.   The  neutral  and  negative 
responses  may  have  been  a  result  of  the  page  design  or  the 
nonsense  text. 

In  addition,  the  findings  of  the  study  suggest  that 
designs  which  do  not  have  a  rule  are  preferred  over  those 
which  do.    Most  subjects'  natural  reading  sequence  was  a  top- 
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down,  left-to-right  approach;  the  insertion  of  a  rule  changed 
this  approach,  which  may  account  for  the  negative  reaction  to 
the  design.  The  results  suggest  that  page  designs  which 
support  the  top-down,  left-to-right  reading  pattern  are 
preferred  over  those  which  disrupt  this  pattern. 

Future  research  should  investigate  several  questions 
raised  by  this  study.   One  question  of  interest  is  whether  an 
additional  study  utilizing  "real"  text  with  various  picture 
placement  would  produce  the  same  preference  results  as  the 
present  study  (Ross  and  Morrison,  1989).    Another  question  of 
interest  is  v/hether  layout  preference  varies  among  poor, 
average,  and  advanced  readers.  Research  on  reading  speed  and 
visual  tracking  imply  that  there  may  be  physiological  support 
for  certain  kinds  of  text  organization  and  placement  of 
illustration  combinations.    An  extension  of  the  existing  study 
should  examine  the  effects  of  unfamiliar  page  designs  on  the 
reading  strategies  of  subjects.    It  is  predicted  that  students 
must  adapt  and  create  new  strategies  when  presented 
unfamiliar  page  designs,  which  may  result  in  increased  reading 
time  and  loss  of  interest. 

Results  of  this  study  provide  instructional  designers 
with  new  information  on  page  design.   Regardless  of  the  page 
design,  the  most  common  natural  reading  pattern  is  a  top- 
down,  left-to-right  sequence.    In  order  to  avoid  negative 
reactions,  decreased  reading  rate,  and  disrupted  thought 
processes,  designers  must  design  pages  which  do  not  interfere 
with  th;s  natural  reading  behavior  of  learners. 
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Table  1 


Percentages  of  Respondents'  Reading  Sequent 


Design 


Pattern 


Top-down,  Left-to-right, 
left-to-right  top-down 


Top-down, 

left-to-right 

top-down 


Design  #1 

91% 

9% 

0% 

Design  #2 

79% 

30% 

0% 

Design  #3 

91% 

9% 

0% 

Design  #4 

65% 

35% 

0% 

Design  #5 

99% 

1% 

0% 

Design  #G 

81% 

16% 

3% 

Design  #7 

97% 

3% 

0% 

Design  #8 

83% 

17% 

0% 
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Figure  1. 
Sample  Page  Design. 
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cere  falele  winse  psano  dy  ser  ciy  tthealif  ouget 
suge  ndis  Thiop  Cousar  li  obguy  tdr  stodma 
pwhs  ih  mg  druakd  urMsb  bguant  owhe  thes 
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unsehat  h  ato  o  n  chse  Iddshasi  efels  s  tino 
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structural  Knowledge 


Knowledge  of  relationships  and  interrelationships  of 
concepts  (ideas) 


Internal 
Connectedness 


Integrative 
Understanding 


Theoretical  Basis  for 
Structural  Knowledge 


In  most  theories  of  LTM.  the  meaning  of  any  concept 
is  determined  by  the  pattern  of  relationships  to 
other  concepts  in  the  learners  knowledge  structure. 


Schema  Theory 
(Rumelhart  & 
Ortony) 


Semantic 
Networks 
(Quillian) 


Active  Structural 
Networks 
(LNR  Group) 
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SemQntIc  Mops 
CousQl  tnteroctlon^ 

Mops 
Concept  hops 
sSplder  hops 


(Concept  Hierarchies^ 
Seniontlc  Feature 
Anolysis 
Cross  ClassUiCQtion 
Toblt$ 


CittgoiictUvy/Rtlited 

Wort3 


Content  Structures 

(Meyer) 
Organizing  Structures 
(Elaboration  Theory) 
Frannes/Slots 
( Armbrusterit  Anderson ) 


Implicit 


Vocabulary  Pre-Teaching 
Advance  Organizers 
Structured  Overviews 
Graphic  Organizers 
Cognitive  Maps 
Digraphs 


txpliclt 


Communicating 
Structural 
Knowledge 


Learning 
Strategies 


Assessing 
Structural 
Knowledge 


Pattern  NoUng 
Semantic  Analysis  of 

Pottem  Notes 
Frome  Gomo 

Generative  Structured  Overviews 
Networking  (Dansereou  et  ol) 
Node  Acqulsltlon/lntegrotlon 
Technique 


Protocol  Analysis  of  Essays 
Word  Association  -  Grophing 
Coonltlve  Hops 
Hlerarchlcol  Trees 
Patterning  NoHqg 
Entailment  Meshes 
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tapBcM  SkMif'M  lif  CMvty^Af  C#QNli^W  Slrvdw^ 


iTMATtO?  DItCltlFTION  A^ntCATtONt  AOVANTAQIt  «t«A0VANTAOC« 


*  tn*arve«e«i 


^fMi««/3toi*                Ftmm  m  mmm             iwemrM  OMifM  •!  tm                  sitte  »rM»M  «•  Wrtf  m^  to  A  $mim  muMMi  •!  ft 

MM  t»  «9Mto  ton  TlMt«  tn  *m     A43|woct  •itrctoM                towHiry  m«  mm.  »wm«  Irw*  M^t 
e  AnMOTMtor  an^  T  M.                «|  liwn,  «Mt  kMM              TMinf  WUBWO 

»r  •«  cMwto*  W                                          UM  ol  Hwili^ilto  »««v««M  eM.  Tltor*  It  Mtori  rMMTtfl 

Mr  wratpto.  OyMMc  Hmw  orfonlM                                    itotoM  iMur*  Mm  t^t$mt  to  «i  «m  liio.iHiii 

GMtoM  ¥/  MMdRf  M»i                                                 cM#tor.  «  Mpto  to  topH.  tIM  •humct  •<  «^  ^^mM 

b*«MM  prtMtftm.                                                         fttnlwctaf  i«r              •*  to  •fimeiuf*  bMvckCA 
Sioto  «•  wmioM  •!  MomMon  wMito                                        oomm  ml 

I  tetofWMow                                             rnmM  Mrf  M»  ow  ta^  to  arfanlM  tfynMue  hmtm  m  rtMMJ 

•  wtwc  »iM|l>ii>  rntt  «M«r«  ftol 

t)«  iwpBitowl  MMtoM  kt  MuM  m  TMi  orfMltmt  atftom*  Mo 

t)«  trmniOM        ffw  «MM  dM«  addrM^  pnmaitf  ■  h*«t 

Ml  *Mrftr  to  MwM  Imrvctton  m  ArtMcai  Mrueiwr*  mim 

•  My  tfrtertt  •(  a  OMioipi).  rttoMM^N  bM«M«n  con- 
c«pto  muoi  tw  MiMddKt 


• 

Pag*  1 


imfMeM  3lrato9tM  lor  C<y><«y«>«  cognatv*  Sfrvcsura 


STRATIOY  DItCmrriON  A^^llCATIONt  ADVANTAQIS  DISADVANTAOf « 

EUtwa*Mi  Thm^cf  Tht  M(*«Ma  al  lM»vcM«.  aM  f        JnMm  Tha  aiabartftoN  tftoory  praaertMo  I.Mto4  aiMrfcal  vaotoca- 

of^anictif  wiMtv  M^MMr  oaMvay  tta  hv>vmuaiicd  in»irtic«M         a  ■Mnfto  to  oeflkptoi  aaqMma  wn  of  ffw  aftoeavtrnM 

n««9a«i«i  u#«rt  tkvawra  al  ffia  »«(aca  Mtor  Coikim  .fOcft  is  r>ypa«iaat<o«  to  bwM  aM  aMoawy  o<  at^aaon 

TVm  t)9M  «f  oantoM  af*  MM  ttabto  cofnMM  akuUjrM  M  aa^ianoM 


tav^  laanwf  ai  ffw  outoM  o<  EUtooniM  aaquanoM  ara 
inatrucion  rtUtvafy  ditfeuii  to 

oonttnta 


but  nalili  aMaaiM  a<  fia  aaMMt 
la  to  liawK.  r«*"*>  liiiaiatin  m 
«to  Htm  Iwaacaiii  to  RKia  (wMai 


Ej^AcM  MaipMaMi  af  ito  aMaratfan 
■taarr  incAjM  mm  a*  ayMtaatwfa  to 

thaw  >i  toainiti»t  mmsm 

ooMtoM.  Tito  i>aiwin  toMM  aap«c«r 
oanvava  «w  airxfctet  al  iw  ooMani  by 
prowtdmo  a  «pad«t  lypa  al  avarvtow 
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DZtAOVUnJICU 


AttVAnc*  0r9«nlx«rt 

0.  AUtUb«l 


Pun>o»«:  to  fAclUtAt*  th« 

Mtntn^ful  v«tt«l 

InXormition. 

on  Autub«l*» 
9ub«u«iptlon  TtMory  of 
MMntnqful  v«rtMl  iMrntng 
vbtch  •U99««t«d  tbat 
co^nlttv*  ttrvctur*  !• 
bt«zarcblc«lly  orgAntscd  In 
t«nM  of  Mfhly  tncluttv* 
coAC*pt«  und«r  vhtch  ar* 
•ub»uji*d  l«tt  inclutlv* 
cooc«ptt  «nd  infonMttoaal 
d«t«. 

i»r«««nt*d  in  «ivanc»  od  tb« 
Mt«rUl-to«b«-l*arn«d  and 
at  A  higher  l«v«l  of 
Abttractlon.  generality  and 
lnclutlv«n«aa. 

Erpotitorv  rorw;  provides 
relevant  proxlMt* 
aubtUMrt.    Ut«d  wb«n 
Mtarlal  It  unfaalllar  and 
no  concept t  ere  knovn  to 
which  new  onea  can  bo 
anchored. 


Adjunct  Inatnjctlooal 
zli  tstl^iitt  fsr  use  In 
•xpoaltory  Idldactlc) 
InttructloA. 

Sett  UiCd  with  Mterlal 
Which  !•  poorly 
organised,  difficult  or 
unf Miliar  to  the  reader 
lo  order  to  locreate  the 
dUcrlmlnabUlty  of  that 
■aterlal. 

Sect  veed  wbeo  the 
learoer'a  ability  U 
lljalted  aa  the  advance 
organiser  facilitates 

the  conttrvctlon  of  an 
Ideational  fraiaework  for 
und^ratandlng  -  a 
weakneaa  for  low  ability 
leamera  (It  abould  be 
noted  th4t  the  research 
evidence  for  thla  Is 
equivocal) . 

Most  effective  In  the 

facilitation  of  long 
tare  retention. 


Theory-bated.  Frovldas 
ManlPefyl  comeat  to 
the  learner  and 
encourages  the  use  of 
that  context. 

C40  take  several  forma 
Including  textual  verbal 
passages,  dlagraiat. 
Illustrations,  concrete 
■odels.  analogies, 
esaagiles.  aets  of  higher 
order  rules,  oral  and 
visual  presentations. 

Effective  with  a  variety 
of  age  groups  and  for  a 
wide  range  of  subject 
fitter. 


No  sttong  evidence  for 
the  efreeclvenese  of 
advance  organisers 
despite  conalderable 
research.    Many  of  the 
■ost  striking  results 
conflict  with  the 
assuiBptlons  of 
9ubsu«ptlon  Theory. 

Advance  organisers  are 
difficult  to  construct 
It  is  seldOM  clear  what 
entails  auterlal  at  a 
'higher  level  of 
abstractnessf 
Incluslveness  or 
generality*  nor  Is  it 
slifile  to  deteraane  if 
the  organ&zer  will  be 
sufficient  to  provide 
the  learner  with  the 
appropriate  "Ideational 
scaffolding*. 


rorm;  Integrates 
new  concofiis  kith  basically 
slnllar  concepts  In 
cognitive  structure.  u»ed 
when  laarners  are  faaililar 
with  Material  to  which  new 
concepts  can  be  anchored. 
Must  clearly  v^ellnette 
ilAilarltles  between  those 
and  mat  trial -to-be- learned. 
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Explicit  Strategies  for  Conveying  Structural  Knowledge 


Crfohlc  Orgarlier: 
Structured  OvervlM 


rascurtioa 

Introduced  as  a  nodlflcatlon 
of  the  advance  organiser  to 
iBiprove  reading  instruction. 

Preaented  In  the  for»  of  a 
tree  dlagrast  to  introduce 
new  vocabulary  to  be  uaed  in 
the  material -to-be-learned. 

Uses  the  spatial 
characteristics  of  dlagraRs 
to  Indicate  the 
relationships  and  distances 
between  key  terms. 


AtyLZcaaricv* 

Vocabulary  Instruction. 

Adjunct  Instructional 
al<i  to  liRprova  the 
co(iv'*benslon  of  textual 
Eater lal. 

Can  be  used  pre-  or 
post-reading.  Research 
indicates  that  the 
latter  Is  nore  effective 
and  that  atructured 
overviews  do  affect 
learning. 


Can  be  uaed  to  activate 
prior  knowledge. 

Students  can  construct 
their  own  structured 
overview*  thus  allowing 
the  technj.qi.e  to  be 
generative  in  nature. 


DltADVUrtACU 

NO  real  basis  m  theory 
Structured  overviews  are 
not  true  advance 
organiser!  at  originally 
clalintd  slnct  they  are 
presented  at  the  sim« 
level  as  the  material- 
to-be-learned. 

Only  preaentt  vocabulary 
to  be  used  in  the 
passage.    No  description 
of  tne  relationship 
between  concepts* 
principles,  etc.,  to  be 
encountered. 


Pictorial  craphlc 
organiser 


r.  Hawk,  N.  HcLeod 
and  0.  Jonatten 


An  extentlon  of  atructured 
overvlewt.    Hose  pictorial 
or  vltual  In  nature.  Focus 
la  on  key  concept t  rather 
than  key  tertit. 

Utea  the  tpatlal 
characterlttlct  of  dlagramt 
to  Indicate  the 
relet lonahlpt  and  dlatancet 
between  key  tenta. 

Participatory  font;  apace  it 
left  for  the  learner  to  fill 
In  key  terms  and  details. 

Final  forw;  the  information 
It  filled  in  for  the 
learner. 


Adjunct  Inttructlonal 
aid  to  provide 
additional  ttructure  to 
the  material -to-be^ 
learned. 

Can  provide  a  graphic 
tynopsls  of  the 
structural 
interrelation shlpa 
between  Ideae  In  the 
materlal-to-be-learned. 

Can  provide  a  post 
organiser  to  suwurlse 
what  has  been  learned. 

Can  be  used  as  a  • 
diagnostic  tool  to  ^ 
assess  student  learning. 


So«e  baals  In  theory; 
i.e.  Autubel's 
Subaut^tlon  Theory  of 
meaningful  verbal 
learning  via  the  procens 
of  Inte^ratlvw 
reconciliation. 

Provides  the  learner 
with  advance  notice  of 
the  atructure  of  the 
mate rial -to-be- learned. 

Provides  a  moderately 
generative  learning 
activity,    an  overall 
structure  la  provided 
but  the  learner  most 
define  temt  tnd  choote 
what  to  nil  In 


Mot  yet  ttrongly 
validated  In  terms  of 
retearch  evidence. 

Require  toms  training  to 
learn  to  construct. 
Tl^-con turning  to 
product.    Alto,  the 
format  la  not  entirely 
clear  and  It  tubject  to 
wide  Interpretation. 

Ko  specific  guidelines 
ibout  the  logical 
ttructure  of  the 
organiser  although 
teveral  aie  suggested; 
e.g  Chronological 
order,  tltrple -to-complex 
tequenc*.  caute/effect 
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explicit  Str«t»9l«t  for  CoAv«ylnq  Structural  ICjtowl«d9« 


Structured  Not*- 
t«Un9 

P.Salth  and  G. 
Toopkint 


crtphic  org<nix«r  in  which 
th«  top  ^•v«l  ttructur*  u«*d 
tiM  .utuiir       lAr«rr«u  &y 
th«  rudor  i«  explicit  in 
tb«  9rtphic. 

Not  «•  pictori«i  In  fonut 
as  th«  pictorial  9raphic 
or9anis«r  Axplainod  abov«. 

Providoa  aovon  or9ani<«r» 
Cor  commott  toxt  ttructur««: 
•  (toacription 

-  tiM/ordar 

-  cau»«/afCact 

-  problaa/aolution 

-  problan/aoiution/raault 

-  co«pariaon/contraat 

-  dafinition/axanplaa 


Adjunct  inatructiortal 
aid  to  provida 

tba  «atarial-to-b«^ 
laarA«d. 

Can  provida  a  graphic 
aynopsia  of  tha 
structural 
intarralationships 
b«tw*«a  idoas  in  tha 
Mtarial-to-^*-laarnad. 

Can  proYida  a  post 
organliar  to  auMMrixa 
What  has  boon  laamad. 

Can  b«  usad  aa  s 

dia9noatic  tool  to 
assaas  stu4ant  laarnin9. 


basis  in  thaoryj 
Auaub«l*a 


•*anin9ful  vaxbal 
laarnin9  via  tha  procass 
of  inta9rativa 
raconclliatloo. 

Can  provida  th«  laarnar 
with  advanca  Dotica  of 
tha  etructura  of  tl>a 
Mtari«l*to-bo-laarnad 
if  prasantad  with  an 
inatructor  9«naratad 
varsion. 

ZC  laarnar  constructadi 
la  an  activity  which  is 
strongly  fanarativa  In 
natura  aa  tha  laarnar 
Must  apply  tha  diCCarant 
scructuras  to  tha 
Mtarial. 


DUADVMtACM 

Appaars  to  ba  no 
rasaarch  avldanca  to 

viiliste  this  r*l«t!v«iw 
racant  tachnlqua. 

Raqulras  tralnln9  to  ba 
usad  aCCactivaly 


(>a9a  3 


Explicit  Strata^las  Cot  Convaylnq  Structural  Knowlad9a 


nStAIXCT 
Concopt  Haps 


J    Novak  and  B. 

Gowin 


Schasvitic  davicas  Cor 
raprasantln9  tha 
relationships  b«tw««n 
concepts  cantarad  around  kay 
topica.  Raprasant 
naanin9Cul  relationships  in 
the  Cora  oC  propositions  (2 
or  aore  concept  labels 
linked  by  worda  In  a 
aanantic  unit) . 

Include  concept  labela  and 
propositional  atataatent) 
alon9  the  links. 

Are  hierarchical  in  format 
aa  they  adhere  to  Auaubel'a 
Subsuffption  Theory  in  which 
hi9her  order  (broader/nore 
inclusivel  concepta  subausM 
new  concepts  or  concept 
■Manin9a  with  nora 
apecXfic/leas  inclusive 
underneath. 

Uae  arrowa  to  indicate  links 
which  are  not 
auperordinste/subosdinate. 


Aa  graphic  organlxers  to 
allow  the  learner  to 
Cocua  on  key  ideas  In  a 
learning  taak. 

As  post  organizer*  to 
provida  a  achaatatic 
suRMary  oC  what  was 
learned. 

Aa  a  creative  activity 
to  foster  recognition  of 
new  concepts. 

A  study  technique  for 
students  to  review 
Mtarial  learned  or 
extract  SManing  Croci 
text. 

An  outlining  technique 
Cor  preparing  a  paper  or 
presentation. 


JUmURACU 

It  learner  conatructed> 
is  an  activity  which  la 
strongly  generative  in 
nature  as  the  learner 
■uat  conatruct  laeaning 
fro«  the  Material. 

Providaa  graphic 
representation  which  can 
be  diacuased  and/or 
altered.    Hovak  and 
Gowin  aee  it  aa  a  tool 
for  negotiating 
awanlnga. 

Subtletlea  ".1  Manlnga 
are  provided  by  linking 
words  can  be  revealing 
oC  student's  cognitive 
crganlxatlon. 

scoring  ayate*  provided 
which  preaents  a  Method 
oC  aaaesalng  the 
algnlflcance  oC  the  Map 
constructed. 


DZ«ADV»TAai 

Hierarchical  nature  may 
be  limiting  and  Corce 
unnecessary  structur*. 

Requires  training  to  use 
eCCectlvely;  Cor 
Instance,  learning  the 
dlCCerence  between 
concept  labels  and 
linking  words*  chooalng 
appropriate  linking 
worda.  learning  to 
eCCectlvely  cluater  tha 
concepta*  etc. 

Are  tisN-conaunlng  to 
construct  aa  the 
developMent  of  synvnetry 
In  the  Map  and  proper 
cluaterlng  oC  the 
concepta  »iy  require 
conalderable  redrawing 
oC  the  Map. 

Linking  words  mat  be 
careCully  choaen  as  the 
•Manlnga  presented  can 
vary  conalderably  with 
the  ter«  selected,  e  g 
the  links  between  the 
concepta  oC  "water"  and 
"ice"  could  labelled  in 
Many  waya  including  "can 
be*.  "becOMea*  *n<i 

"SOMtlMeS  is*. 

Scoring  systef*  require* 
subjective  )udg>nants  ot 
Kteanlngfulnass  and 
validity  of  concept 
relationships  and  cross 
links. 
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Irpllclt  Str«t«9ltt  for  Convtyln?  structural  Ki)owl«d9« 


Coqnltlv*  Kipt 


dUkhoff 


Inttrvc'.or  ••l*ctt  k*y 
concAfttf  torMf  l«port«nt 
•v«fltt  oz  I0««t  from  th« 
knowl«^t  doMln.    Thtt*  art 
•  rr«aq«<l  into  p«lrt  In 
Mtrlx  for*.    Tht  pairs  ar* 
AtslqiMd  •  r«tln9  fron  1  (no 
rtl«tlofl«blp)  to  9  (vory 
■tronq  rtlatlonahlp)  vblch 
It  convtrtcd  to  d«clMl 
totm. 

A  prlnclplt  coMpOMntt 
«(ulytlt  It  p«rfonM<l  vhlch 
tr«ntl«ttt  tht  rtlAtlontblp 
jud^Mntt  to  <U«taDctt 
(hiqhly  rtUtod  id»«t  cloa* 
toq«th«r  and  Ittt  rtl«t*d 
ontt  aott  distant} . 

Tht  dlttanett  art  ua*d  to 
crtatt  the  "cognltlvt  m»p'  - 
a  graphic  array  of  conctpt 
points  In  apact. 


Aa  ^r^phlc  or^anlxtr^  to 
allow  tht  iMrotr  to 
foeua  on  kty  Itfaaa  In  a 

iMrnlnf  taak. 

At  poat  orfaitlztra  to 
provldt  a  achaiMtlc 
suHury  of  what  waa 
ltarn*d. 


Frovlda  a  rtvltw  of 
prior  knowltdqt  for 
atudtnta. 

AaalfnMnt  of  "atrtngth 
of  r«latlonahlp"  ratlnga 
allowt  atatlatlcal 
analyala  of  tht  aapa. 

ThU  haa  l*d  to 
validation  of  tht 
ttchnlqut  Indlcatlnq: 

1)  tnhanc^d  atudtnt 
taaay  tcor«t. 

2)  Incrtattd  rtllablllty 
uf  atudter  qtnarattd 
jud^Mnta. 

3)  IncrtatAd  aladla^lty 
b«tw««n  atudcnt 
rtlatlonahip  jud9«»ntt 
and  thost  of  tht 

Inatnictor. 


OUAOVMrUCU 

Provldt  a  rtlatlvtly 
paaalvt  Inttractlon. 
Not  hlqhly  qtittratlvt  In 
aaturt. 

T«cbnlqut  rtqulrta  a 
fairly  tophlatlcattd 
atatlatlcal 

manipulation.  Co^nltlvt 
aapa  art  fairly 
difficult  to  9tntratt 
and'  htnct*  not  ustful 
aa  a  ttudy  ttchnlqvt  or 
aa  M  t»chnlqut  for 
prtparlnq  paptra, 
prtatntatlona*  ttc. 

Jud^Mnt  of  tht  atrtnqth 
of  rtlatlonahip  btt^atn 
tht  conctpt  a#  ttrtu* 
ttc.<  la  arbitrary.  Ho 
quldtllnta  provided 
conctrnlnq  how  to  judqt. 

NO  Indication  of  tht 
naturt  of  tht 
rtlatlonahip . 


Pagt  S 
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stratify 

STRUCTURAL  HHPS 


Pattern  Notes 


Ottcrjptlon 

Graphically 
reor«seflt 
relationships 
between  concepts 
or  Ideas. 


Oroanlzes 
Inforaatlon  In  a 
relational  unner 
showing  linkages 
between  topics. 


Structural  Knowl<Kif« 
ilAkUm  STRATEGICS 


Procmliire 


•begin  with 

central  Idea 

blocked  In 

center  of 

blank  page. 
•free*«ssociate 

about  the  subject 

addlna  lines 

conttinino 

related  ideas, 
•keep  elaborating 

on  these  second- 
ary Ideas. 


Vses 

•note-taking 

•erg4n1ie/r<fir«*f!- 

Ke  infonaatlon 
•Identify  l^^r- 
t^nt  concepts 
(Ideas)  and 
relationships 

•study  technique 
(relationships/ 
self-atsessMnt) 

•outlining  tech- 
nique (note- 
taking  and 
response 
structuring) 


Advajitages 


Olsadvanta^s 


•easily  learned 
by  a  variety  of 
learners 

•allows  for 
niltlPle  rela- 
tionships 

•less  constraining 
■ore  Individual 
(unique) 


do  not  depict  the 
nature  of  the 
relationship 
(except  Inferred) 


Sewintlc  Analysis 
of  Pattern  Notes 


Post-hoc 
classification 
of  the  links 
(seaantlc  rela- 
tionships) between 
Ideas  on  a  pattern 
note. 


eppllcatlon  of  an 
aigorltha  (series 
of  questions) 
resulting  In  the 
classification  of 
the  seaantic 
relationship 
between  Ideas 


to  Identify  the 
types  of 
relationships 
between  the 

concepts  (Ideas) 
In  a  pattern  note 


allows  exploration 
of  the  nature  of 
the  relationships 
between  Ideas  In  a 
pattern  note 


not  easily  used  by 
young  learners 


Structural  Cnowl«dgt  LMrnlng  Strattgl«t 


Strategy 
Concept  Haps 


Description 


Graphically 
displays 
relationships 
between  concepts 
In  a  hierarchical 
fashion. 


Procedure 


•extract  concepts 
fro«  Material 

•identify 
relationships 

•Mp  top  down 


Oses 


•self-assessMnt 

•analytical  too^. 
•  for  extract*^^ 
atJkR^ng  froa 
text,  notes,  also 
for  understanding 
literary  worth 

•facilitates  para- 
phrasina  . 

•notetaking 

•response- 
structuring 


Advantages 


•depicts 
relationships 
b^itween  concspts 

•proaotes 
•saningful 
understanding 


Disadvantages 


constraining 
than  pattern  notes 


Networking 


A  spatial 
organization  of 
Mterial  Into 
node-link  aaps  or 
networks. 


•  identify  i^K)r- 
tant  concepts  or 
ideas  (nodes)  in 
the  Mterial 

•represent  their 
interrelation- 
ships (links) 
using  a  set  of 
n««M  links  to 
code  relation- 
ships 


.^outlining 
technique 

•orga  n  1  z  1  ng/nian  1  p- 
uUting  problem 
space 

•facilitate 
probles-solving 


•assists  in 
viewing  overall 
concept 

•  links  MV  aid  in 
retrieval 


wre  constraining 
than  pattern 

noting 


Analysis  of 
Key  Ideas 


ERIC 


Structural 
alternative  to 
networking  aided 
by  worksheets 
which  specify 
categories  of 
<ief1n1t1on  and 
co^)ar1son. 


•Identify  key 
ideas/concepts 

•develop  systOMt- 
ic  definitions 
and  elaborations 
of  the  concepts 

•Interrelate 
important  pairs 
of  concepts 


•(sec  networking) 
•elaborate, 
coifiare  concepts 


helpful  for 
learners  requiring 
structure  or  who 
have  Halted  study 
skills 


mre  structured/ 
constraining 


30^04 


Structuml  Knowl«dM  UArnlng  Strat«gl«s 


Strittgy 

NAIT 

(the  Nod« 
Acquisition  and 
Integration 
Technique) 


Description 

A  structured 
sesantlc  network- 
ing technique. 


Procedure 

•Identify  Impor- 
tant concepts 

•describe  the 
links  between 
concepts 

•coapletlon  of 
definition 
worksheets  which 
require  learners 
to  Identify  six 
types  of  rela- 
tionships for 
each  key  concept; 
characteristics 
and  descriptors, 
antecedents, 
consequences, 
"vidence,  sub- 
^ts,  supersets 

•comparison  work- 
sheets (coapare/ 
contrast 
concepts) 


Uses 

•production  of 
mre  complete 
concept  defini- 
tions 

•Integration  of 
Information 


Advantages 

•learners  produce 
more  varied  and 
complete  defini- 
tions 

•learners  Inte- 
grate new  Infor- 
mation 

•aids  In  Identi- 
fying whet  Is 
most  Important 
In  a  passage 


Ditadvantaget 

•  tiwe 
•training 


ERIC 


304 

305 


ASSESSING  KNOWLEDGE  STOUCTURES 
StraUgy  Dttcrlptlon 


Ums 


AdvantSQM 


DItidvtnttgtt 


Conttnt 


RtadkigTIm* 
Ttsti 


rtKlno  limt  It  friMiurid 
and  oofTtlaltd  wWi  th« 

numlMr  of  pcopotltions. 


Construct  ValUtty 
of  propotltlons 


EasIV  Adr»lni$t«r«d 


UtuAliy  Nmltfd  to 
Itboratofy  ut«.  Timtng 


Propoilltont 


Stmantic  Racafl 
Tfsti 


Sublteti  art  glytn  para- 
graph. Laiar  ara  givtn 
aammaa  and  ara  aiMd 

wtiattwr  or  not  ttwy 
hava  saan  tha  tamanoe 
batora. 


Construct  Validity 
ofsamantic 
thaodasof 
mamory 


EasBy  Administerad 


Usually  MmHad  to 
latx>raiofy  usa. 


Propositions 
artdsamwtk: 
thtorias 


Hiararchlcat 
Timing  Tasts 


Maasurad  whh  timing 
laats  using  oonoa(>ts  at 
various  lavals  of 
ganarality. 


Construct  validity     Easily  Admlnstarad    Usually  HmHad  to  HIararchlcal 

of  hIararcfUcal  laboratory  usa.  oonoapt 

mamory  Timing  Davicas  art  structura 

naadea 


Pattam  Notas 


Subfccts  ara  askad  to 
Iraa  asaodala  on 

papartoagkon 

word.  Sooradby 
oouTi'Jng  numbar  of 
HnKs  batwaan  v^ords. 


oomplax  i(nowiadga 
structures 


EasVy  Admlnlstarad 
and  can  ba 
Intarpratad 

Intultivaly 


Unks  (ralation- 
shipa)  batwaan 
ooncaptsara  not 
labaiad 


Structure  of 
schema 
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ASSESSING  KNOWLEDGE  STRXTTURES 
Strategy  Daacrtptl^n 


Uaas 


Advantagaa 


Diaadvantagea 


Content 


Free  Word  Subjects  are  given  a  word  Assess  cognitive 

Association  and  are  askad  to  Hst  the  structure  and 

Tests  first  word  that  oomes  correlate  with 

to  mind.  Below  it,  the  content  structure 

second  word,  etc. 


Easily 

Administered 


Ralatiortships 
are  not  expiained 


Structure  of 
schema 


Controlled  Subjects  are  gVeti  a  list  Ass^oognWve  Easily 

Word  of  words  and  asked  to  rank  structure  and  Administered.  Can 

Association  order  them  according  to  corralate  wHh  be  Interpreted 

Tests  which  worda  are  most  content  structura  intuitively 

related  in  meaning. 


Relationships 
are  not  expiained 


Structure  of 
schema 


Tree 

Construction 
Tasks 


SUbfecta  construct  a 
linear  graph  m  whtoh 
the  Intarsadlona  are 
worda  and  the  connecting 
Hnes  are  relationshipa 


Assess  cognltl/e 
structure 


Easily 

Administered 


Relationshipa 
are  not  explained 


Structure  of 
schema 


Concsptmapa 


Subjects  are  aaked  k>  tink 
ooncepia  that  relate  and 
label  the  Inks.  A  scoring 
system  exists  which  can 
relate  map  to 
content  stn)Cture. 


Assess  oognlthw 
structure  and 
correlate  with 
content  structure 


Labeled  Unks  make 

relationships 

explicit 


Scoring  system 
is  ftnked  k> 
content  structure 


Structure  of 
schema 
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ASSESSING  K^X]WL£OGe  STRUCTURES 
StrattQy  Dttcrlption 


AdvantflQM 


Olta^vantagti 


Contant 


MamtMrthlp 
D«dsion  Tasks 


Sublacts  ara  tknad  as 
thay  dadda  wiMttar  an 

action  ia  Invofvad  in 
parformInQ  a  glvan 
activity. 


Asaasamantof 
contaniiotan 

indivMuarf 
script  Build 
script  'norms* 


Easily 

adminlstarad 


Ail  possft>tiitlas 
iTUCt  ba  praparad 


Scripts  or 
Sowma 


Q#mKatino 
Examples 


Sublets  ara  givan  a 
caiagocy  and  aakad  lo 
ganarala  membars  or 
Inslanoas  of  tha 
catagory. 


Judoasamanlic 

proximity 


daasand 
axamplas 


Etslty 

administered 


May  Interact  with 
genera}  abHHy  to 
genaraie  exarnplas 


Semamtc 
proximity  of 
concepts 


Conceptual 
Models 


Concept  maps  ol 
content  structure. 


Assess  content 
atructure 


Easily 

administered 


Olsagraemeni  among 
experts  In  loossly 
denned  fields 


Content 
Structure 


Concept  Typicality 
Ratings 


Sub|ects  are  given  a 
auperordlnate 
caUgory  and  are  asked 
to  rate  the  typlcayty 
of  given  instances. 


Judge  semantic 
proximity 


daasand 

examples 


Easily 

administered 


Many 

possibilities 
must  be  prepared 


Semantic 

proximity  of 
concepts 


Subtraction  TImo  to  complete  a 

Tectmiqje  taak  Is  compared  witti 

time  to  complete  a 
simiar  task  with  one 
element  varied.  Assumed 
that  dWerenca  is  caused  by 
different  processing. 


information 
processing 


UsaWaway  to 

assessacoven 

system 


Laboraiofy 
meaaurement  Is 
necessary.  Relies  on 
inferencM  as  to  actual 
processes  involved 


Productions 
end  production 
systems 


ASSESSING  KNOWLEDGE  STRUCTURES 
Strategy  Deecrlptlon 


Ueee 


Advantagee 


DIaadvantagee 


Content 


Actual  Tasks 


Subjects  perform  tasks 
that  require  automate 
perfonr  ance. 


Infers  the  existence 
of  productions 
that  underlie 
performance 


Easily  judged 
visually 


Reieson 

logic  of  inference 


Productions  or 
Exr;utive  routine 


Think-Atoud 
Tasks 


S(i}|ect  !s  asked  to 
speak  akMJd  thoughts 
as  he  carries  out 
actMty  or  solves 
a  problem. 


Assess  protMm 
solving  strategies 
or  production 
systems 


Makes  covert 

activities 

visible 


Speaking  aloud 
may  Interfere  with 
though  processor 
may  Inhibit  natural 
processes 


Productions, 
production  systems, 
or  problem 
solving  strategies 


Diagnostk:  Tests 


Carahitly  created 
tests  that  measure 
components  of  a  process 
rather  than  Just  the 
outoomo. 


Assess  missing  steps 
of  a  production 
system 


Are  rutural  to 
learner  If 

introduced  early 
In  instruction 


Productions 
and  productkm 
systems 


Scanning  Tlir,« 


Subjects  are  tkned  as 
they  scan  'mental 
pictures.' 


Construct  vaiktlty 
of  plctortai 
memory 


l.aboraiory  timing 
required 


Pk:tori8l 
memory 


Mazes 


Lab  animals  are 
shown  various 
routes  and 
atlemattves  paths 
to  aooompUsh  goals. 


Assess  presenco  or  Visually 
abeer>oe  of  cognitive  scored 
mapa 


Questk)nable  axtemal 
vaUdity  as  applied 
to  humans 


Cognitive  maps, 
pictorial  memory, 
episodk:  memory 
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ASSESSING  KNOWLEDGE  STRUCTURES 
Strategy  Duacrlptlon 


AdvantaQM  DIaadvantaoaa 


Contant 


Hun)ank4«m 


Subjects  ara  put  through 
a  dioriamino  situation  and 
Shanaikadtopolnttoa 
spadnc  object  or  a  otnarai 
dirtction 


Ass«is  spaiirl 
abiltlts  and 
sansao! 

diraction 


Easily 

adminlstarad 


Spatial  knowtidga 
uns«  of  diraction 


RaUand 
Ground  lasts 


Subiacts  sit  in  darlianad 
room  in  movabto  chair  va 
ara  askad  to  crlant 
ihamsalvsa  wfth  a  horizon 


To  deiarmina  M6 
dapandanoa  or  fMd 
kidipandanoa 


RapNcabla  and 
rsaHstic  rasuits. 
Corrtlation  to 
aknplar  tasts,  such 


Complax  tasUng 
aquipmant  rtquirad 


ftguras  tast 


FMddipendanoa 
FWd  lndepend«rx:a 
or  spatlsi 
knowladQa 


Mental  Rotation 
Tasks 


Sublact  ia  glvan  two 
plduraa  of  an  obfact  and 
aikod  to  datarmlna 
whathaf  or  no(  tha  ob^d 
l«  tha  first  pictum  can  ba 
rotatad  to  look  Mia  tha 
aacondona 


Construct  validity 
of  spatial  or 
pictorial  mamory. 
iLtaasurafnam  of 

pictorial  sKiiis 


Points  to  otherwisa 
hiddan  abiUty 


May  raquire  timing 
dsvicas 


Pictorial  0/ 
spatial  kTJwIsdga 
or  sHill% 
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Abstract 


Automation  has  impacted  virtually  every  manufacturing 
and  information  operation  in  the  world.  Instructional 
design  (ID)  is  no  exception.  In  this  article,  we 
suggest  three  basic  metaphors  for  automating 
instructional  design  activities,  each  typifying 
different  kinds  of  systems.  Within  this  framework,  we 
describe  several  prototype  systems  designed  to  automate 
different  functions  of  the  ID  process.  Prospects  for 
future  systems  are  discussed,  including  criteria  for 
evaluatinq  automated  ID  systems,  and  the  need  for 
continuing  research  in  ID  problem-solving  activities. 
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Analyzing  Automated  Instructional  Design  Systems: 
Metaphors  from  Reiateu  Design  Professions 


The  instructional  design  (ID)  process  comprises  the 
procedures;  decisions ;  and  knowledge  necessary  to  plan, 
develop ;  and  evaluate  instructional  systems  and 
programs.  Actual ly,  the  process  itself  can  be  seen 
historically  as  a  step  toward  automation  and  control 
of  a  complex,  costly,  and  diffuse  enterprise.  It  grew 
out  of  efforts  in  the  1950s  and  60s  within  m  litary  and 
university  settings  to  define  systematic  methods  for 
engineering  instructional  products ,  drawing  on  ideas 
from  general  systems  theory,  psychology,  and  management 
science.  The  thinking  was  that,  at  a  time  when  systems 
design  methods  were  being  used  to  develop  sophisticated 
weapons  and  manufacturing  products,  might  the  same 
general  design  principles  be  used  in  the  design  of 
complex  training  systems?  Models  were  developed  that 
proceeded  systematically  from  instructional  needs  and 
goals  toward  design  of  products.  Continuing  evaluation 
and  testing  were  core  features  of  these  models,  as  was 
the  idea  that  all  parts  of  an  instructional  system 
should  fit  together  and  contribute  to  the  goals  of 
instruction.  Today  dozens  of  different  ID  models  are 
being  used  to  guide  the  design  and  development  of 
training  products  in  a  variety  of  public  and  private 
sector  organizations. 

The  next  major  chapter  in  the  history  of  ID  models 
is  presently  being  written  as  ID  models  are  being 
incorporated  more  completely  intc  integrated  computer- 
based  systems  for  design  and  production.  Presently, 
several  such  prototype  systems  are  being  developed  that 
further  automate  ID  processes;  we  will  report  on  some 
of  those  systems  in  this  chapter,  and  provide  reference 
information  on  other  systems.  An  additional  purpose 
of  the  chapter  is  to  analyze  different  kinds  of 
automated  systems,  to  suggest  appropriate  roles  for 
these  systems,  and  to  point  to  future  directions  and 
needed  research. 

One  way  of  thinking  about  the  problem  is  to  compare 
the  ID  process  to  similar  manufacturing  or  programming 
design  processes.     Gibbons  and  0*Neal  (1988a)  state: 

Instructional  design  belongs  to  the  same  family  of 
processes  that  creates  building,  car,  and  widget 
design. . .During  both  design  processes,  the  goals 
of  design  turn  into  functions  to  be  performed  by 
the    product,    which    ultimately    materialize  as 
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features.  This  links  features  directly  to  original 
design  goals*  Documentation  of  the  design, • .moves 
from  general  expressions  of  intent  and  form  to 
detailed  expressions  of  function  and  finally  to 
exacting  specifications  for  individually  catalogued 
products • 

Something  can  be  learned  by  examining  design  models  in 
engineering  and  other  design  fields.  Autoiuated 
approaches  used  in  these  fields  can  serve  as  metaphors 
for  ID  systems.  In  the  section  below ,  we  discuss  three 
different  design  metaphors  t)  at  serve  as  a  framework 
for  describing  automated  ID  systems:  (1)  computer-aided 
design  and  manufacturing  (CAD/ CAM)  systems ,  (2)  expert 
system  advisor  systems ;  and  (3)  computer-aided  systems 
engineering  (CASE)  tools  used  in  programming  and 
information  systems  design. 


The  CAD/CAM  Metaphor:  ID  Environments 

CAD/CAM  systems  include  graphics  design  and  data 
manipulation  tools  that  facilitate  the  engineering  and 
design  of  products.  The  flexibility  of  these  tools 
allows  engineers  the  freedom  to  conceptually  try  out 
different  solutions  and  quickly  and  easily  make 
adjustments  in  a  product  *  s  structure  or  dimensions . 
The  engineer  using  CAD/ CAM  tools  has  more  power  and 
flexibility  than  predecessors  using  traditional  tools 
and  instruments. 

Not  surprisingly,  CAD/CAM  tools  have  drastically 
changed  the  engineer's  job  requirements  over  the  last 
10  years.  Their  dramatic  rise  in  use  has  virtually 
eliminated  the  need  for  traditional  draftsmen ;  drawing 
tables ,  squares ,  and  compasses .  These  changes ;  of 
course ;  have  been  reflected  in  schools  of  engineering 
around  the  world ,  and  in  the  individual  engineer '  s 
continuing  professional  education.  CAD/CAD  tools  do 
not  replace  the  engineer's  design  expertise;  rather 
they  provide  a  fertile  problem-solving  environment  for 
professionals  to  do  their  jobs  better. 


The  IDioM  System 

An  ID  tool  designed  to  emulate  a  CAD/CAM  tool  would 
be  designed  for  professional  instructional  designers 
in  a  way  that  would  allow  considerable  flexibility  and 
support.  As  an  example ,  consider  the  IDioM  system 
developed  for  Apple  Computer ,  Inc.  (Gustafson  &  Reeves , 
1988) IDioM  is  essentially  a  shell  or  superstructure , 
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written  in  HyperCard,  that  oversees  ID  processes  on  a 
Macintosh  computer «  Within  IDioM;  instructional  design 
is  conceptualized  as  having  7  main  functions,  each  with 
several  sub-functions  or  activities.  The  7  functions 
and  their  associated  activities  are  shown  in  Table  1. 


Insert  Table  1  about  here. 


The  IDioM  system  makes  full  use  of  an  array  of 
Macintosh  applications,  such  as  Excel,  More,  and 
PageMaker.  At  any  point,  the  designer  may  be  within 
a  tool  appl ication — say  MacPro j ect  II ,  but  is  also 
thought  to  be  within  a  specific  IDioM  function  and 
activity,  such  as  select  media  within  the  Design 
Function.  The  designer  has  access  to  three  special 
menus:  the  general  IDioM  Menu,  the  Manage  Menu,  and  the 
Help  Menu.  The  IDioM  Menu  provides  orienting  and 
guidance  information  for  working  within  the  system  and 
its  various  functions  and  activities.  The  Manage  Menu 
includes  further  guidance  on  planning  and  reporting 
progress  to  others.  The  Help  Menu  provides  information 
needed  for  navigating  through  the  system  and  connecting 
to  the  next  logical  tool  or  activity.  The  Help  Menu 
also  allows  the  recording  of  personal  comments,  for  use 
in  editing  and  reviewing  work. 

The  IDioM  system  seems  to  fit  a  CAD/ CAM  type  of 
design  model  because  it  is  designed  primarily  to 
address  the  needs  of  professional  designers.  It 
imposes  a  certain  structure  on  development  by 
incorporating  basic  ID  concepts,  but  still  allows 
considerable  flexibility  in  specifics  of  design  and 
production.  The  designer  has  available  a  number  of 
data-organizing  and  production  tools  to  use  at  any 
point  in  the  design  process .  Help  and  support  is 
provided,  but  not  in  a  way  to  severely  constrain  the 
designer  *  s  personal  style .  The  IDioM  system  is  now 
being  called  ID  Library  by  Apple,  and  is  being  beta 
tested  at  6  public  and  private  sector  locations 
(Gustafson  &  Reeves,  1990) . 


The  Instructional  Design  Data  Base 

Another  prototype  that  seems  to  fit  the  CAD/ CAM 
model  is  the  Instructional  Design  Data  Base  developed 
by  designers  at  WICAT  Systems  (Gibbons  &  O^Neal , 
1988b) .  This  product  is  meant  to  eventually  provide 
ID  tools  that  encompass  the  entire  ID  process. 
Presently  a  Task  Analysis  Interface  has  been  developed 
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for  MS-DOS  computers  (Gibbons,  O^Neal,  &  Monsori;  1988) 
that  allows  designers  to  gener^ate  either  text -based  or 
graphic  hierarchical  analyses.  These  analyses  are 
stored  within  the  system  and  may  be  sorted;  searched; 
and  modified.  Although  our  understanding  of  the  system 
is  based  on  the  operation  of  the  Task  Analysis 
Interface  and  written  descriptions  of  the  larger 
system;  the  key  to  the  larger  system  is  the  idea  of  an 
ID  database.  Instructional  designers  work  through  a 
successive  series  of  representations  of  content — needs 
statements;  task  analyses;  goals  and  objectives;  test 
items ;  instructional  displays ;  etc .  All  of  these 
content  representations  are  interrelated  and  should  be 
linked  together.  Changes  in  one  data  type  should  be 
reflected  in  changes  in  the  entire  systCTi.  This  core 
notion  of  database  drives  the  design  and  planned 
capabilities  of  the  overall  system. 

The  CAD/ CAM  metaphor  is  apt  in  describing  the 
Instructional  Design  Data  Base  because  the  system  is 
aimed  at  the  professional  designer  and  offers  a  measure 
of  flexibility  within  the  constraints  of  the  database. 
The  object  of  the  system  is  to  provide  a  productive 
environment  within  which  the  designer  can  perform 
design -related  tasks  in  an  efficient  and  productive 
manner.  Presently;  the  ID  database  concept  is  being 
incox-porated  more  completely  into  another  proprietary 
WICAT  product.  No  information  is  presently  available 
concerning  this  new  product. 


Expert  Systems  Tools:  ID  Advisors 

Expert  systems  are  decisionmaking  tools  that  make 
use  of  reasoning  mechanisms  developed  by  artificial 
intelligence  researchers  (Grabinger;  Wilson;  & 
JonasseH;  in  press)  .  Most  expert  systems  have  a  system 
of  IF-THEN  rules  that  constitute  the  knowledge  in  a 
narrow  domain.  These  rules  comprise  the  system's 
"knowledge  base;"  which  is  then  used  to  arrive  at 
judgments  about  what  to  do  in  a  given  situation.  The 
computer  program  thus  becomes  an  "advisor"  to  the  user; 
recommending  a  course  of  action  or  choice  based  on  the 
available  data.  Expert  system  advisors  typically 
engage  the  user  in  a  consultation  in  which  the  user 
responds  to  questions  by  providing  information  about 
the  specific  case  in  hand.  The  advisor  applies  that 
information  to  its  knowledge  base  and  returns  with  a 
recommendation  or  decision.  Successful  expert  systems 
have  been  developed  for  decisionmaking  tasks  such  as 
diagnosing  an  illness ;  choosing  a  site  to  drill  a  well; 
or  troubleshooting  a  complex  piece  of  equipment  (Harmon 
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and  King,  1985) . 


Another  way  of  thinking  about  expert  systems  is  as 
an  intelligent  job  aid.  Job  aids,  in  the  form  of 
checklists,  flowcharts,  or  step-by-step  instructions, 
are  written  tools  or  helps  to  support  job  performance; 
that  is  also  essentially  the  rcle  of  expert  system 
advisors  (Welsh  and  Wilson,  1987) .  On  this  view, 
expert  system  advisors  do  not  provide  a  complete 
working  environment  for  doing  the  entire  job;  instead, 
they  are  occasionally  consulted  in  addressing  specific 
tasks  where  the  user  needs  special  support  or 
expertise.  Job  aids  may  be  used  by  professionals  in 
working  with  seldom-'encountared  or  complex  tasks,  but 
more  frequently  are  designed  for  the  novice  worker  who 
needs  additional  reminding  and  support  for  specific 
tasks.  As  the  novice  continues  on  the  job  and  performs 
the  task  repeatedly,  the  need  for  the  job  aid  may 
disappear. 


Park  Row  ID  Tools 

Park  Row  is  a  software  development  and  pviblishing 
firm  founded  by  Greg  Kearsley,  who  has  developed  a 
series  of  stand-alone  expert  system  rule  bases  to 
perform  a  variety  of  instructional  design  tasks.  The 
Behavioral  Objectives  system  prompts  the  user  to 
provide  information  about  expected  outcomes  which  it 
then  assembles  into  an  objective.  The  system  then 
queries  the  user  to  analyze  the  ob jecti .  *^  and  then 
classifies  it  using  Bloom's  taxonomy  of  objectives. 
Problem  Analysis  provides  advice  on  which  of  several 
data  gathering  techniques,  such  as  fault  tree  analysis, 
interviews,  focus  groups,  etc.,  to  use  in  performing 
a  job  ^inalysis.  CBT  Analyst  leads  the  user  through  a 
series  of  decisions  to  detemnine  whether  or  not  to 
develop  computer-based  training.  Cost/Benefits 
Analysis  queries  the  user  for  information  about  a 
prospective  training  situation .  The  information 
then  sent  to  a  spreadsheet  to  estimate  the  costs  and 
benefits  of  developing  training.  Most  of  these  systems 
were  written  in  Turbo  Pascal  and  run  in  an  MS-DOS 
environment  with  minimal  memory  requirements. 

The  Park  Row  tools  fit  clearly  into  the  advisor  or 
intelligent  job  aid  category.  Each  tool  provides  only 
analysis  and/or  advice  about  the  ID  process,  rather 
than  an  actual  production  environment.  The  tools  may 
be  found  useful  by  professional  designers,  but  would 
more  specifically  help  the  novice  designer  or 
technician  in  need  of  greater  structure  and  expertise. 
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CASE  TOOLS:  Computer-aided  ID 

Within  the  last  5-10  years,  a  category  of  software 
tool  has  attracted  attention  among  systems  designers 
and  programmers.  CASE  (computer-aided  systems 
engineering)  tools  are  programs  that  automate  the 
programming  process  (Bigelow,  1988),  sometimes  referred 
to  as  code  generators  or  applications  generators.  In 
concept,  a  user  would  use  a  CASE  tool  by  specifying  the 
inputs  and  desired  outputs  for  a  given  product,  along 
with  other  constraints,  and  the  computer  would 
automatically  generate  fully  written  code  for  that 
product.  Although  that  goal  sounds  idealistic,  some 
progress  has  been  made  in  developing  systems  that 
compile  code  to  meet  specified  needs.  The  idea  is 
similar  to  a  programmer  who  compiles  code  blocks  from 
a  library  of  routines  rather  than  starting  from  scratch 
on  a  new  project.  Much  of  the  progreunming  work  has 
already  been  done  and  is  available  in  the  form  of 
routines;  the  job  of  the  programmer  is  to  assemble  and 
fine-tune  these  routines  into  a  smoothly  working 
program  that  solves  the  information/performance  problem 
at  hand. 


ID  Expert 

Using  the  idea  of  CASE  tools  for  instructional 
design,  we  could  imagine  a  product  that  inputs 
information  about  the  context,  content,  learner,  etc., 
and  then  automatically  compiles  a  specific 
instructional  plan  or  product.  ID  Expert,  although  in 
its  initial  stages,  is  a  system  that  philosophically 
bears  some  resemblance  to  this  notion.  ID  Expert  is 
being  developed  for  the  Army  Research  Institute  by  M. 
David  Merrill  and  Zhongmin  Li  of  Utah  State  University 
and  Human  Technology,  Inc.  of  McLean,  Virginia  under 
contract  with  the  U.S.  Office  of  Personnel  Management, 
Office  of  Training  Development.  It  is  a  combined 
consultation  and  production  system  which  provides 
analysis  of  content  and  instructional  strategies.  In 
the  current  phase  of  development,  it  is  implemented  in 
Hyper Cr ad  from  Apple  and  Nexpert  from  Neuron  Data  for 
the  Macintosh  family  of  computers.  The  current  version 
is  being  tied  to  the  Research  Reference  Interface,  a 
hypertext  system  that  provides  reference  information 
on  the  theory ,  pol icy ,  empirical  support ,  and  logic 
used  by  ID  Expert.  Like  ID  Expert,  RRI  is  screen- 
oriented  and  menu-driven  and  is  written  in  HyperCard. 
ID  Expert  consists  of  several  modules: 
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o  an  information  gathering  module  that  queries 
the  user  on  content  to  be  taught  and  relevant 
learner  characteristics  (such  as  motivation 
level  and  prior  knowledge) ; 

o  a  reasoning  model  that  uses  the  information 
gathered  to  create  and  reason  with  rules  in 
Nexpert's  inference  engine; 

o  a  specification  module  that  presents  advice 
on  course  and  content  structure  and  the 
required  instructional  strategies; 

o  an  authoring  module  for  creating  instructional 
sequences;  and 

o  an  explanation  module  which  explains  the  rule 
base  and  connects  to  RRI. 


ID  Expert  begins  by  querying  the  designer  about  the 
title,  audience  attributes,  the  goal  and  its 
attributes,  and  recommends  an  appropriate  content 
structure.  Next  the  system  queries  the  designer  about 
specific  content  attributes  and  elaborates  the  content 
structure  while  recommending  a  course  organization, 
instructional  strategies,  and  individual  instructional 
transactions.  These  transactions  can  then  be  created 
and  sequenced  in  the  authoring  module. 

Unlike  most  other  automated  ID  systems,  ID  expert 
builds  in  considerable  theory.  Based  on  elaboration 
theory  for  its  course  level  strategies  (Reigeluth  & 
Stein,  1983)  and  component  design  theory  for  developing 
micro-level  strategies  (Merrill,  1987) ,  the  system 
offers  designers  much  more  specific  and  concrete 
support  for  selecting  instructional  strategies  based 
on  type  of  learning  outcome  than  is  available  in  other 
systems.  It  is  in  this  sense  that  ID  Expert  resembles 
programming  CASE  tools.  The  designer  supplies 
information  about  a  given  problem,  then  ID  expert  is 
meant  to  compile  the  pieces  into  a  coherent 
instructional  strategy.  In  a  sense,  ID  Expert's 
ambition  is  to  automate  the  process  to  the  point  that 
the  computer  provides  essential  expertise  for  designing 
instructional  products,  leaving  less  discretion  in  the 
hands  of  individual  designers. 

Returning  to  our  discussion  of  CASE  tools  for  a 
moment,  although  the  idea  of  automatic  code  generators 
obviously  has  appeal,  forecasts  and  promises  of  such 
tools  have   exceeded  actual   delivery  performance  of 
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tools  developed  to  date.  The  problem  is  complex, 
particularly  because  programs  must  be  developed  in 
specific  contexts  with  constraints  that  are  often 
unique.  This     means     that     prograns  produced 

automatically  following  a  generic  pattern  must 
inevitably  be  adjusted  and  changed  to  fit  the  new 
situation.  The  fine-taning  and  changing  that  must  be 
done  often  approach  in  complexity  the  task  of 
programming  from  scratch.  The  ambition  of  a  program 
like  ID  Expert  may  face  similar  obstacles.  The  state 
of  research  sind  theory  in  instructional  design  may  not 
be  able  to  hold  up  the  platform  of  decisionmaking  upon 
which  ID  Expert  is  built*  Although  very  explicit  and 
guided  systems  such  as  ID  Expert  are  a  tremendous  boon 
to  theory  development,  their  practical  use  may 
encounter  problems  similar  to  those  of  programming  CASE 
tools • 


Mixing  the  Metaphors:  Hybrid  Environments 

To  this  point,  system  types  have  been  described  in 
idealized  teras.  Of  course,  many  systems  cross  over 
and  overlap  our  metaphors.  The  IDioM  system,  for 
^*xample,  contains  a  connections  option  off  the  Help 
Menu  that  lends  itself  wr 1 1  to  expert  systems 
technology.  The  system  could  track  where  the  designer 
is  within  IDioM  and  anticipate  tiie  next  needed  activity 
or  tool.  Thus  within  an  overall  production  envirorinent 
for  instructional  design,  expert  systems  can  easily 
have  a  place. 

The  same  thing  can  be  said  for  strongly  theory- 
based  products.  Different  theories  might  be  thought 
of  as  templates  to  be  superimposed  upon  a  system:  At 
the  designer's  request,  a  different  model  or  template 
could  be  selected  to  drive  the  design.  Thus  the 
rigidity  of  any  particular  approach  is  much  less 
threatening  to  the  professional  designer,  because  it 
can  always  be  rhrown  out  and  replaced  by  another. 

It  would  seem  desirable  that  a  design  environment 
combine  the  best  elements  of  production  tools 
(graphics,  text,  and  video),  database  and  organization 
capabi 1 it ies ,  and  expert  systems  or  hypertext 
technologies  where  appropriate.  Each  of  the  systems 
discussed  above  exhibits  different  strengths  in  these 
areas.  In  the  section  below,  we  turn  to  our  own 
prototype  system,  the  Instructional  Design  and 
Development  (IDD)  Advisor.  Presently  the  system  is 
designed  primarily  as  an  advisor  and  front  end  to  a 
database;  future  plans  call  for  its  integration  with 
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production  tools  to  provide  a  more  complete  development 
environment. 


IDD  Advisor 

The  ir^  Advisor  is  an  on-going  project  designed  to 
provide  intelligent  job  aid  support  for  assisting  an 
instructional  designer,  teacher,  or  course  writer  to 
follow  the  instructional  design  and  development  (IDD) 
process.  The  IDD  Advisor  is  also  an  intelligent  front 
end  to  a  database  containing  the  components  of  the 
instructional  sequence  being  developed,  so  that  as 
instructional  decisions  are  made,  the  results  of  those 
decisions  are  stored  in  a  database. 

The  IDD  Advisor  is  a  structured  rule-based  expert 
system  with  several  rule  sets.  The  arrangement  of 
these  rule  sets  is  illustrated  in  the  context  tree  for 
the  IDD  Advisor  in  Figure  1.  Each  node  represents  a 
separate  rule  set  which,  in  effect,  functions  as  a 
separate  knowledge  base.  Each  rule  set  draws  on  fairly 
generic  and  well-established  ID  concepts. 


Insert  Figure  1  about  here. 


The  first  rule  system  encountered  is  the  executive 
menu,  which  queries  the  user  to  ascertain  if  thir  is 
a  new  design  project.  If  the  project  is  new,  it 
executes  the  project  analysis  rule  system.  If  the  user 
is  adding  onto  an  existing  project,  the  executive  menu 
enables  the  user  to  select  the  rule  system  that  she  or 
he  needs  to  work  in. 

For  new  projects,  the  user  would  typically  work 
through  a  project  analysis  rule  set  first.  This  rule 
set  collects  information  about  the  project  and 
establishes  whether  the  project  is  instigated  by  a 
performance  problem  or  whether  it  involves  new  systems, 
automatic  training,  or  educational  develv^pment.  If  the 
project  is  predicated  on  a  performance  problem,  the 
project  analysis  module  will  access  the  performance 
analysis  module.  The  user  will  provide  information 
about  the  nature  of  the  performance  problem.  For 
instance,  a  corporate  user  might  identify  employee 
productivity  as  a  problem.  If  the  project  is  not 
problem-based,  the  project  analysis  module  will  access 
the  needs  assessment  rule  set.  Regardless  of  the 
purpose  of  the  project,   the  project  analysis  module 
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saves  information  about  the  project. 

The  performance  analysis  rule  set  leads  the  user 
through  a  typical  performance  analysis:  identify  the 
goal  performance  state,  the  current  state,  determines 
the  discrepancy,  determines  the  cause  and  classifies 
the  cause  of  the  performance  problem.  If  the  problem 
is  caused  by  motivational  or  environmental  problems, 
the  user  would  exit  the  system.  The  performance 
analysis  rule  base  would  help  the  user  identify  the 
cause  and  whether  or  not  training  is  the  proper 
solution.  In     the     future,      motivational  and 

environmental  problem  rule  sets  may  be  developed.  If 
the  problem  is  determined  to  be  a  skill/knowledge 
problem,  then  the  problems  are  recorded  and  indexed  in 
the  database,  and  the  user  is  led  to  the  task  analysis 
module. 

The  needs  assessment  rule  set  is  accessed  if  the 
project  analysis  determines  that  the  project  is  not 
problem-based.  If  that  is  the  case,  users  need  to 
decide  which  data  collection  instrxment  is  appropriate 
for  conducting  the  needs  assessment.  Should  they  use 
a  questionnaire,  interview,  focused  group  or 
brainstorming  session?  Users  are  led  through  the  needs 
assessment  technique  which  helps  them  decide  why  type 
of  instrument  is  appropriate  for  their  need.  Having 
determined  the  instrxment  type,  they  need  to  select  the 
question  form  and  question  type  in  order  to  construct 
the  questions.  The  outcome  of  these  modules  is  a  needs 
assessment  instrument  with  the  questions  identified. 
This  information  is  then  stored  in  the  database.  After 
collecting  the  data,  users  would  access  the  needs 
assessment  interpretation  rule  set  to  help  them 
interpret  the  outcome.  The  users  would  then  be  led  to 
the  task  analysis  module. 

The  task  analysis  process  has  three  rule  sets  that 
assist  the  user.  The  first  module  helps  the  user 
select  the  most  appropriate  task  analysis  technique 
from  a  list  of  27  found  in  Jonassen,  Hannum  &  Tessmer 
(1989).  A  sample  of  the  rules  in  this  rule  set  is 
illustrated  in  Figure  2.  The  technique  rule  set  may 
access  the  tool  rule  set  to  determine  the  most 
appropriate  information  gathering  tools.  Finally,  the 
task  analysis  rule  set  will  help  users  to  interpret  and 
analyze  the  information  they  collect.  The  sub-tasks 
will  be  stored  in  the  database. 


Insert  Figure  2  about  here. 
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Following  the  task  analysis,  the  objective  writing 
and  test  item  writing  rule  sets  will  access  the  tasks 
stored  in  the  database  and  help  the  users  to  convert 
them  t'o  ob j  ectives ,  classify  those  ob j ectives ,  and 
write  test  items  that  measure  those  objectives. 

Having  developed  the  test  items,  designers  would 
then  access  the  instructional  design  modules,  which 
would  help  them  to  select  the  necessary  presentation 
forms  and  instructional  strategies  for  the  sequence  of 
tasks/ objectives  determined  in  the  previous  modules. 
Again,  the  rule  sets  will  interact  with  the  database 
by  pulling  the  information  out  of  the  database  and 
using  it  to  help  the  users  to  select  the  instructional 
lesson  components. 

Finally,  designers  could  access  the  delivery  module 
to  access  advice  on  selecting  the  appropriate  delivery 
system  for  presenting  the  instructional  components. 
The  system  may  be  expanded  to  include  a  message  design 
module  in  the  future. 

The  prototype  knowledge  bases  run  under  VP  Expert 
from  Paperback  Software.  VP  Expert  is  a  structured, 
rule-based  system  with  considerable  power  and 
flexibility.  The  information  base  is  being  created  in 
dBase  Ill-f  from  Ashton-Tate.  VP  Expert  can  interface 
with  dBase  files  in  order  to  function  as  an  intelligent 
front  end  to  the  database.  Both  software  packages  run 
on  MS-DOS  personal  computers  (IBM  or  compatible) .  The 
reasons  for  selecting  the  software  packages  currently 
being  used  to  develop  IDD  Advisor  are  pragmatic.  They 
are  inexpensive,  readily  available,  and  commonly  used. 
That  makes  the  system  more  useful  to  a  wider  range  of 
users. 

Harmon  (1986)  describes  a  seven  step  process  for 
design,  developing,   and  implementing  expert  systems. 

1)  Front  end  analysis  -  performance  problem  and 
cost  benefit 

2)  Task  analysis 

3)  Prototype  development  -  development  of  a  small 
version  to  test  feasibility  and  desirability 
of  system 

4)  System  development  -  develop  entire  system 

5)  Field  test  the  system 
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6)  Implementation  -  disseminate  the  system 

7)  Maintenance  -  upgrade  the  system  and  provide 
user  support 

The  IDD  Advisor  is  currently  at  the  fourth  step  in  this 

process    system  development.      The  task  analysis 

yielded  the  context  tree  shown  in  Figure  1.  A 
prototype  of  two  of  the  rule  sets  was  demonstrated 
(Jonassen,  1988) .  The  remainder  of  the  rule  sets  and 
the  database  filers  need  to  be  developed. 


Needed  Research  on  Instructional  Design 

All  of  the  systems  described  above  incorporate  ID 
concepts  and  procedures  that  have  proven  their  worth 
over  the  past  25  v  ars  in  field-based  development 
pro  j  ects .  At  the  same  time ,  automating  these 
procedures  affords  a  new  opportunity  to  validate  the 
effects  of  these  procedures.  With  the  computer  to 
assist  in  data  gathering  and  storage,  differences 
between  systems  can  be  explored.  This  in  turn  can 
refine  our  knowledge  base  regarding  effective  ID 
procedures. 

While  generic  ID  models  have  been  somewhat 
validated  by  practical  experience/  there  is  a 
surprising  lack  of  empirically  derived  knowledge  about 
the  problem-solving  heuristics  of  professional 
instructional  designers.  How  do  individual  designers 
do  their  work  within  the  overall  ID  framework?  How 
closely  do  they  follow  prescriptions  from  the  models; 
when  do  they  use  their  intuition  and  professional 
judgment  to  override  conventional  ID  recommendations? 
What  kinds  of  learning,  management,  and  design 
principles  do  they  actually  incorporate  at  various 
points  in  the  design  process?  Vir1:ually  no  formal 
research  has  been  completed  to  study  these  questions, 
yet  an  understanding  of  designers '  problem-solving 
strategies  seems  extremely  relevant  to  the  design  of 
automated  ID  systems.  We  also  need  to  be  concerned 
about  designers'  use  of  tools:  How  do  designers  use 
computers  and  other  reference/production/organization 
tools  in  their  work?  The  designer/computer  interface 
needs  to  be  more  thoroughly  studied  before  automated 
ID  systems  can  be  optimized. 

Rather  than  be  stymied  by  our  lack  of  knowledge  in 
these  areas,  we  should  see  the  advent  of  automated  ID 
systems  as  opportunities  to  conduct  in-depth  studies 
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and  examine  processes  in  closer  detail.  Thus,  in  the 
long  run,  automated  ID  systems  will  not  only 
incorporate  our  present  knowledge  of  ID,  but  also 
contiribute  to  that  3cnowledge  in  the  future.  In  the 
same  way  that  cognitive  science  researchers  develop 
computer-based  models  of  cognitive  processes, 
instn?ctional  designers  will  come  to  view  the 
development  of  automated  ID  systems  as  a  valuable 
method  of  research  and  a  way  to  contribute  to  our 
knowledge  base  about  instruction. 
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Table  and  Figure  Captions 

Table  1.  IDioM  Functions  and  Activities  (from  Gustafson 
&  Reeves,  1988) . 

gure  2.  Context  tree  for  the  IDD  Advisor.  Shaded  boxes 
are  presently  not  developed. 

Figure  3.  Sample  rules  from  the  IDD  Advisor. 
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FUNCTION 


ACTIVITY 


ANALYZE 


DESIGN 


DEVELOP 


EVALUATE 


IMPLEMENT 


Conduct  needs  and  goal  assessment 
Define  learner  characteristics 
Conduct  task  analysis 
Examine  existing  materials 
Conduct  financial  analysis 

Specify  objectives 
Sequence  objectives 
Determine  strategy 
Select  media 
Prepare  assessment 

Define  lesson  treatment 
Outline  content 
Prepare  training  materials 
Integrate  interactive  media 

Collect  project  documentation  data 
Assess  worth  of  training  objectives 
Conduct  formative  evaluation 
Assess  immediate  effectiveness 
Evaluate  training  impact 
Estimate  cost  effectiveness 

Prepare  implementation  plan 
Communicate  with  Training  Center 
Support  implementation 
Train  the  trainer 


PRODUCE 


Prepare  component  list  and  estimate 

Prepare  packaging 

Specify  and  order  components 

Assemble  package 

Debrief 


MANAGE 


Estimate  project  scope 
Start  up  with  project 


Table  1.  Functions  and  Activities  from  the  iDioM 
environment,  presently  called  the  ID  Library  (from 
Gustafson  &  Reeves,  1988) . 
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Figure  2 

Rules  for  Selecting  Task  Analysis  Techniques 


IF  Type-Job  AND 

Scope-Single  AND 

Function-Sequence  AND 

CosUTime-LovAND 
  Expertise-High 

THEN  Technique-LetrningJCdntingency...Analysis; 


RULE  12 

IF  Type-Job  AND 

Scope-Single  AND 
Function-Description  AND 
CosL-Time-LovAND 

  Expertise-Lov 

THEN  Technique-BehaviondLAnaiysis; 


RULE  13 

IF  Type-Job  AND 

Scope-Single  AND 

Function-Description  AND 

CosL.Time-HighAND 

Expertise-High 
THEN  Technique-FunctionikLJobJ^&lysis: 


RULE14 

IF  Type-Job  AND 

Scope-Single  AND 

Function-Description  AND 

CosUime-LovAND 

Expertise-High 
THEN  Technique-Letrning-Contingency-^alysis: 


RULE  13 

ff  Type-Job  AND 

Scope-Single  AND 
Function-Description  AND 
Cost.Time-LovAND 
Expertise-Lov 

THEN  Technique-MethodSLAnalysis: 


RULE  16 

IF  Type-Job  AND 

Scope-Single  AND 
Function-Description  AND 
CostJrime-LovAND 
Expertise-Lov 

THEN  Technique-TaskDescription: 
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providing  students  with  an  opportunity  to  practice  newly 
acquired  skills  and  knowledge  is  an  important  component  in 
designing  an  instructional  strategy.    While  many  instructional 
design  theories  include  recommendations  for  designing  practice 
activities,  Salisbury,  Richards,  and  Klein  (1985)  point  out  that 
most  of  these  theories  fail  to  address  how  to  design  practice 
that  is  motivational. 

A  number  of  educators  argue  that  instructional  games  are 
effective  for  providing  motivating  practice  of  newly  acquired 
skills  and  information.    These  scholars  argue  that  instructional 
games  are  motivational  because  they  generate  enthusiasm, 
excitement,  and  enjoyment  and  because  they  require  students  to  be 
actively  involved  in  learning  (Coleman,  1968;  Ernest,  1986;  Rakes 
&  Kutzman,  1982;  Wesson,  Wilson,  &  Mandlebaum,  1988).  Other 
scholars  argue  that  instructional  games  decrease  student 
motivation.    These  authors  believe  that  the  motivational  aspects 
of  instructional  games  are  limited  to  those  who  win,  and  that 
losing  an  instructional  game  produces  a  failure  syndrome  and 
reduces  self  esteem  (Allington  &  Strange,  1977;  Andrews  &  Thorpe, 
1977) . 
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While  theorists  argue  about  the  motivational  aspects  of 
instructional  games,  researchers  have  investigated  the  effect  of 
using  games  on  student  motivation.    The  results  of  these  studies 
are  inconclusive.     Some  researchers  report  that  the  use  of 
instructional  gaming  increases  student  interest,  satisfaction, 
and  continuing  motivation  (Devries  &  Edwards,  1973;  Sleet,  1985; 
Straus,  1986) .     Others  report  that  playing  a  game  does  not 
influence  student  satisfaction  or  attitude  toward  school  (Devries 
&  Slavin,  1978).     Investigators  also  report  that  instructional 
games  influence  school  attendance.    Allen  and  Main  (1976)  found 
that  including  instructional  gaming  in  a  mathematics  curriculum 
helped  to  reduce  the  rate  of  absenteeism  of  students  in  inner- 
city  schools.     Studies  by  Raia  (1966)  and  Boseman  and 
Schellenberger  (1974)  indicated  that  including  games  in  a  college 
business  course  has  a  positive  affect  on  course  attendance  but 
not  on  expressed  interest  and  satisfaction. 

In  addition  to  the  possible  motivational  benefits 
instructional  games,  educators  believe  that  games  are  effective 
for  helping  students  learn.     Scholars  argue  that  instructional 
games  make  practice  more  effective  because  students  become  active 
participants  in  the  learning  process  (Ernest,  1986;  Rakes  & 
Kutzman,  1982;  Wesson,  Wilson,  &  Mandlebaum,  1988).     Others  argue 
that  games  foster  incorrect  responding,  are  an  inefficient  use  of 
instructional  time,  and  that  the  rate  of  practice  in  a  game 
cannot  compare  to  a  flashcard  drill  or  reading  a  connected  text 
(Allingtoij  &  Strange,  1977;  Andrews  &  Thorpe,  1977). 
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Researchers  have  attempted  to  answer  whether  instructional 
games  are  an  effective  method  for  learning.     The  results  of  these 
studies  are  inconclusive.     Some  investigators  report  that 
instructional  games  are  effective  for  assisting  students  to 
acquire,  practice,  and  transfer  mathematical  concepts  and  problem 
solving  abilities  (Bright,  1980;  Bright  &  Harvey,  1982;  Bright, 
Harvey,  &  Wheeler,  1979;  Devries  &  Slavin,  1978;  Dienes,  1962; 
Rogers  &  Miller,  1984).     Others  report  that  using  an 
instructional  game  to  practice  math  skills  assists  slow  learners 
but  not  more  able  students  (Friedlander ,  1977) .    Research  on  the 
use  of  instructional  games  in  college  business  courses  has 
produced  inconclusive  or  nonsianif icant  finding  in  many  studies 
(Bosenian  &  Schellenberger ,  1974;  Greenlaw  and  Wyman,  1973;  Raia, 
1966)  ,  while  instructional  games  have  positively  influ'jnced 
learning  in  actual  business  training  settings  (Jacobs  &  Baum, 
1987;  Pierfy,  1977).    Even  advocates  of  instructional  gaming 
believe  that  there  is  some  disagreement  over  whether  games  teach 
intellectual  content  and  skills  (Boocock,  1968) . 

There  are  several  explanations  for  the  inconsistent  findings 
from  research  concerning  the  effect  of  instructional  games  or. 
motivation  and  learning.    One  is  that  much  of  the  research  on 
instructional  gaming  has  been  conducted  using  flawed  experimental 
designs  and  methods  (Reiser  &  Gerlach,  1977;  Remus,  1981;  Stone, 
1982).    Another  explanation  is  that  many  of  the  studies  on 
instructional  gaming  have  not  investigated  the  integration  of 
games  in  an  instructional  system.    Gaming  advocates  suggest  that 


games  should  be  used  with  other  instructional  methods  such  as 
lecture  and  textbooks  (Clayton  &  Rosenbloom,  1968) .    A  third 
explanation  is  that  researchers  examining  the  effect  of 
instructional  gaming  on  motivation  have  not  adequately  defined 
and  operational ized  the  variable  of  motivation.    After  an 
extensive  review  of  instructional  gaming,  Wolfe  (1985)  indicated, 
"no  rigorous  research  has  examined  a  game's  motivational  power, 
[or]  what  types  of  students  are  motivated  by  games"  (p.279) . 

The  purpose  of  this  paper  is  to  describe  the  results  of  a 
study  conducted  to  determine  the  effects  of  using  an 
instructional  game  on  student  motivation  and  performance. 
Motivation  was  defined  using  the  ARCS  model  of  motivation 
(Keller,  1987a) .    This  model  suggests  that  motivation  in  an 
instructional  setting  consists  of  four  components:  attention, 
relevance,  confidence,  and  satisfaction.      We  hypothesized  that 
students  using  an  instructional  game  to  practice  newly  acquired 
information  would  indicate  that  this  method  enhanced  their 
attention,  relevance,  confidence,  and  satisfaction.     In  addition 
since  the  study  was  designed  integrate  the  game  into  an 
instructional  system,  we  attempted  to  determine  the  effect  of 
using  a  supplemental  reading  on  student  motivation  and 
performance . 

Method 

Subjects 

Subjects  were  seventy-five  underyraduate  education  majors  a 
a  large  southwestern  university.    The  students  were  enrolled  in 
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required  course  in  educational  psychology  and  participated  in  the 

study  to  satisfy  a  course  requirement* 

Materials 

Materials  used  in  this  study  were  an  instructional  game  and 
a  worksheet,  both  designed  to  provide  practice  of  information  and 
concepts  presented  in  a  lecture,  the  textbook  Essentials  of 
Learning  for  Instruction  by  Gagne  &  Driscoll  (1988) ,  the 
Instructional  Materials  Motivation  Scale  (Keller,  1987b) ,  and  a 
measure  of  performance. 

The  instructional  game  was  developed  by  the  authors  in  order 
to  provide  subjects  with  practice  on  the  information-processing 
model  of  learning.     The  instructional  game  consisted  of  a  game 
board  that  graphically  represented  the  information-processing 
model  of  learning,  a  direction  card  that  explained  the  rules  of 
the  game,  and  a  set  of  25  game  cards.     Each  game  card  had  a 
practice  question  about  the  information-processing  model  of 
learning  on  the  front  and  feedback  with  knowledge  of  correct 
results  on  the  back. 

The  worksheet  was  also  developed  by  the  authors  to  provide 
subjects  with  practice  on  the  information-processing  model  of 
learning.     The  worksheet  was  four  pages  in  length  and  included 
the  same  25  questions  that  appeared  on  the  game  cards.  After 
subjects  completed  a  set  of  five  questions,  the  worksheet 
instructed  subjects  to  turn  to  the  last  page  for  feedback. 

The  Instructional  Materials  Motivation  Scale  (IMMS) , 
developed  by  Keller  (1987b),  was  used  to  measure  student 
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perception  of  the  motivational  characteristics  of  the 
instructional  materials.    The  IMMS  includes  four  subscales  to 
measure  the  motivational  components  of  attention,  relevance, 
confidence,  and  satisfaction.    Keller  (1987b)  reported  that 
Cronbach's  alpha  reliability  of  this  instrument  is  .89  for 
attention,  .81  for  relevance,  .90  for  confidence,   .92  for 
satisfaction,  and  .96  for  overall  motivation. 

A  15  item  constructed  response  posttest  was  used  to  measure 
student  performance.    The  items  on  this  posttest  were  developed 
by  the  authors  to  determine  subject  mastery  of  the  information- 
processing  model.    The  internal  consistency  reliability  of  this 
measure  was  .77. 
Procedures 

All  of  the  subjects  attended  a  lecture  on  the  information- 
processing  model  of  learning  and  were  told  to  read  chapter  two  in 
the  textbook  Essentials  of  Learning  for  Instruction  by  Gagne  & 
Driscoll  (1988).    Two  days  later,  subjects  were  randomly  assigned 
to  either  a  treatment  or  control  group.    Subjects  in  both  groups 
were  given  thirty  minutes  to  practice  the  information  presented 
in  the  lecture  and  assigned  reading  by  using  either  the 
instructional  game  or  the  worksheet. 

Treatment  group  subjects  used  the  instructional  game  to 
practice  the  information-processing  model.    After  being  informed 
that  they  would  be  playing  a  game,  subjects  were  randomly  placed 
in  groups  of  eight  to  ten  and  were  asked  to  form  two  teams  of 
players.     Each  group  received  the  game  materials  described  above 
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and  the  experimenter  read  the  game  rules  aloud.     Subjects  were 
given  thirty  minutes  to  play  the  game. 

subjects  in  the  control  group  used  the  worksheet  to  practice 
the  same  items.     Each  subject  worked  individually  for  thirty 
minutes  to  complete  the  worksheet.    Subjects  were  told  to  review 
incorrect  items  if  time  permitted. 

Upon  completion  of  the  practice  activity,  all  subjects 
completed  the  Instructional  Materials  Motivation  Scale  and  then 
took  the  posttest.     Subjects  were  also  asked  if  they  had  attended 
the  lecture  on  the  information-processing  model  and  if  they  had 
completed  the  assigned  reading  from  the  textbook. 

Results 

Motivation 

Multivariate  analysis  of  variance  (MANOVA)  was  used  to  test 
for  an  overall  difference  between  groups  on  the  motivation 
scales.    An  alpha  level  of  .05  was  set  for  the  MANOVA  tests. 
These  analyses  were  followed  by  univariate  analyses  on  each  of 
the  four  IMMS  subscales.     In  order  to  account  for  the  possibility 
of  inflated  statistical  error,  alpha  was  set  at  .025  for  the 
univariate  analyses  using  the  Bonferroni  method  (Stevens,  1986). 

Results  indicate  that  using  the  instructional  game  to 
practice  information  had  a  significant  effect  on  motivation.  A 
significant  MANOVA  effect,  F(4,   64)  =  6.57,  £  <  .001  was  found 
for  the  treatment  on  the  motivatior  measures.  Univariate 
analyses  revealed  that  subjects  who  played  the  game  rated  this 
method  of  practice  as  motivational  in  the  four  areas  of  attention 
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Z(l,  67)  =  21.91,  E  <  -001'  relevance  1(1/  67)  =  15.05,  E  <  .001, 
confidence  £(1,  67)     =  16.30,  E  <  .001,  and  satisfaction  Zd,  67) 
=  24.71,  E  <  '001.     Table  1  includes  a  summary  of  means  and 
standard  deviations  on  each  motivation  subscale  for  the  game  and 
the  non-game  groups. 


Table  1  about  here 


Results  also  suggest  that  completion  of  reading  assignment 
had  an  influence  on  motivation.     A  significant  MANOVA  effect, 
F(4,  64)  =  2.94,  E  <  -05,  was  found  for  this  variable  on  the 
motivation  measures.     Follow-up  univariate  analyses  revealed  that 
the  motivational  area  of  confidence  was  significantly  affected  by 
completion  of  reading  assignment  Z(l,  67)  =  6.52,  E  <  .025. 
Attention,  relevance,  and  satisfaction  were  not  significantly 
influenced  by  completion  of  reading  assignment.     Means  and 
standard  deviations  can  be  found  in  Table  1. 
Performance 

Performance  was  measured  using  a  15  item  constructed 
response  posttest.     Analysis  of  variance  (ANOVA)  was  used  to  test 
for  differences  between  groups  on  the  performance  measure.  An 
alpha  level  of  .05  was  set  for  all  statistical  tests. 

Results  indicate  that  completion  of  assigned  reading 
significantly  contributed  to  posttest  performance,  F(2,  72)  = 
14.87,  E  <  -001.     subjects  who  indicated  that  they  had  read  the 
assigned  materials  significantly  performed  better  on  the  posttest 
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that  those  who  did  not  complete  the  reading  (see  Table  1) . 
While  results  suggest  that  subjects  : i  the  treatment  condition 
outperformed  control  group  subjects  on  the  posttest,  this 
difference  was  not  statistically  significant. 

Discussion 

The  major  purpose  of  this  study  was  to  determine  the  effect 
of  using  an  instructional  game  on  student  motivation  and 
performance.     The  results  of  the  study  suggest  that  vsing  an 
instructional  game  as  a  method  of  delivering  practice  does 
enhance  the  motivation  of  students  in  the  fou::  areas  of 
attention,  relevance,  confidence,  and  satisfaction.    However,  the 
results  show  that  an  instructional  game  does  not  necessarily 
contribute  to  enhanced  performance  whe.a  it  is  used  to  practice 
information.     These  results  occurred  perhaps  occurred  for  several 
reasons. 

The  instructional  game  used  in  this  study  provided  students 
with  a  visual  representation  of  the  information-processing  model 
of  learning  and  required  them  to  be  active  participants  in  the 
teaching/learning  process.     Keller  (1937a)  indicated  that  visual 
representations  and  active  participation  are  two  strategies  that 
can  increase  student  attention  in  an  instructional  setting. 
Furthermore,  it  is  possible  that  using  the  game  contriLuced  to 
the  results  found  for  attention  because  of  a  novelty  effect. 
Some  researchers  report  that  student  motivation  and  interest 
fluctuate  and  decrease  as  the  novelty  effect  of  a  game  wears  off 
(Dill,  1961;  Greenlaw  &  Wyman  1973),  while  others  report  that 
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interest  tends  to  persist  over  time  in  gaining  settings  (Dill  & 
Doppelt,  1963) .    While  novelty  may  be  a  reason  for  increased 
attention  in  this  study,  this  explanation  should  be  considered  as 
positive  by  instructional  designers  who  are  concerned  with 
providing  motivating  practice  to  students.    Motivation  and 
attention  can  be  increased  when  variability  and  novelty  are  used 
in  the  classroom  (Keller,  1983,  1987a). 

The  results  found  in  this  study  for  the  motivational  factor 
of  relevance  is  consistent  with  theory  proposed  by  gaming 
advocates.    These  authors  argue  that  students  will  not  question 
the  relevance  of  educational  content  when  it  is  presented  via  an 
instructional  game  (Abt,  1968;  Rogers  &  Miller,  1984).  In 
addition,  Keller  (1983)  indicated  that  instructional  designers 
can  make  instruction  motivational  by  designing  materials  that  are 
responsive  to  the  needs  of  students.     Orbach  (1979)  indicated 
that  games  are  excellent  methods  to  motivate  students  with  a  high 
need  for  achievement  because  a  game  can  include  an  element  of 
competition.     Orbach  (1979)  also  theorized  that  games  can 
motivate  students  with  a  high  need  for  affiliation  when  the  game 
requires  interaction  among  individuals  and  teams.  The 
instructional  gar^e  used  in  this  study  included  a  moderate  level 
of  competition  and  required  students  to  interact  through  the  team 
approach. 

The  instructional  game  used  in  this  study  also  provided 
circumstances  for  student-directed  learning.    As  a  motivational 
strategy,  atudent-centered  learning  has  been  linked  with 
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increased  confidence  (KeJler  &  Dodge,  1982).     The  finding  that 
the  game  increased  student  confidence  is  consistent  with 
theorists  who  suggest  that  games  can  influence  student  efficacy 
(Abt,  1968)  and  researchers  who  .-eport  that  students  rate  the 
task  of  gaming  as  less  difficult  than  other  instructional 
techniques   (Devries  &  Edwards,   1973) . 

The  positive  finding  for  satisfaction  is  also  consistent 
with  theory  and  research.     A  number  of  scholars  indicate  that 
instructional  games  contribute  to  motivation  becau3e  they  pro^'ide 
intrinsic  reward  and  enjoyment  (Coleman,  1968;  Ernest,  1986; 
Rakes  &  Kutzman,  1982) .     Researchers  report  that  instructional 
games  lead  to  increases  in  student  satisfaction  (Devries  & 
Edwards,  1973;  Strauss,  1986).     The  results  of  this  study  support 
theorists  and  researchers  who  suggests  that  students  enjoy  the 
gaming  approach  in  instruction. 

While  tne  results  did  suggest  that  instructional  games  have 
a  strong  effect  on  student  motivation,  the  game  used  in  this 
study  did  not  have  a  significant  impact  on  student  performance. 
However,  subjects  completing  an  assigned  reading  significantly 
performed  better  and  had  more  confidence  about  their  perfor-iance 
than  those  who  did  not  complete  the  reading.     These  results  ma- 
have  occurred  iue  to  the  nature  of  the  reading.     Even  though  all 
the  students  were  provided  with  necessary  concepts  and 
information  in  a  lecture,  the  textbook  Essentials  of  Learning,^ 
Instruction  by  Gagne  &  Driscoll   (1988)  provides  readers  with 
practice  and  feedback  on  the  ideas  presented  in  each  chapter. 
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This  additional  practice  and  feedback  more  than  likely  influenced 
both  the  performance  and  confidence  of  those  who  completed  the 

assigned  reading. 

The  findings  of  this  study  have  some  implications  for  the 
design  of  practice.    While  many  instructional  design  theorists 
indicate  that  students  should  be  provided  with  an  opportunity  to 
practice  newly  acquired  skills  and  knowledge,  most  fail  to 
address  how  to  design  practice  that  is  motivational  (Salisbury, 
Richards,  &  Klein,  1985).    The  results  of  this  study  suggest  that 
instructional  designers  can  provide  students  with  a  motivating 
practice  alternative  that  is  as  effective  as  more  traditional 
methods  of  practice.     Instructional  designers  should  also  include 
reading  assignments  that  provide  additional  practice  in  their 
instructional  strategies  to  increase  student  performance  and 
confidence  about  that  performance. 

AS  was  done  in  this  study,  future  research  should  integrate 
instructional  games  into  a  system  to  determine  if  this  method  has 
an  impact  on  educational  outcomes.     Besides  using  a  game  as 
practice,  research  could  be  conducted  to  examine  the  effect  of 
using  a  game  to  present  other  instructional  events  such  as 
stimulating  recall  of  prior  knowledge  or  as  a  review  of  learning. 
Researchers  of  instructional  gaming  should  continue  to 
investigate  the  effect  of  using  a  game  on  student  motivation  and 
should  be  specific  in  their  operational  definition  of  motivation, 
implementation  of  these  suggestions  •.;ill  assist  us  in  determining 
how  to  design  practice  that  is  both  effective  and  motivational. 
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Table  1 

]>/TAans  and  P<-andard  Deyj^^-^'^^^  Atiiention  <h)  Fel eyapg^  (R) 
rnnfidence  (c\  Satisf antii on  (S)  and  PerfOCTnaPf^^>  fP)  Measures 


Treatment 

Game 
n=37 

4.22 
(0.58) 

3.71 
(0.58) 

4.06 
(0.57) 

3.88 
(0.86) 

10.49 
(3.20) 

Non  Game 
n=38 

3.77 
(0.89) 

3.13 
(0.69) 

3.31 
(0.90) 

2.72 
(1.02) 

9.39 
(3.60) 

Text  Reading 

Read 
n=40 

3.81 
(0.65) 

3.49 
(0.57) 

3.91 
(0.67) 

3.34 
(0.89) 

11.25 
(3.07) 

Not  Read 
n=35 

3.74 
(0.&4) 

3.34 
(0.68) 

3.44 
(0.61) 

3.27 
(0.99) 

8.45 
(3.22) 

Note.    Maximum  scores  =  5.00  for  A,  R,  C,  S  and  15.00 
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Viouai  Discrimination,  Learning,  and  Page  Layoui:  Wrap-Around, 
Run-Around,  and  Transparent  Text 


As  desktop  publishing  revolutionizes  printing  and  publication, 
sophisticated  computer  programs  offer  the  novice  a  wide  range  of 
techniques  formerly  available  only  to  professional  layout  artists.  Now 
anyone  can  design  and  produce  their  own  Instructional  materials  by 
transforming  existing  art  into  professional  quality  work  that  can  be  pasted 
into  any  variety  of  text  formats. 

Several  popular  software  programs  are  noted  for  versatility  and 
ease  of  layout  design.  They  not  only  feature  mn-around  text,  but  now 
boast  wrap-around  text  and  transparent  text  features  as  well.  Run- 
around  text  is  that  which  is  "boxed"  around  the  graphic,  maintaining  a 
square  border  around  the  visual  (see  Figure  1 ).  Wrap-around  text  is  that 
which  flows  smoothly  around  the  graphic  and  "hugs  it  tight",  leaving  an 
irregularly  shaped  border  (See  Figure  2).  Transparent  text  is  that  which 
has  a  graphic  laid  over  it;  the  reader  reads  the  text  through  the  graphic 
(See  Figure  3).  These  formatting  features  are  described  by  software 
publishers  as  desirable  software  advancements,  which  allow  the 
designer  to  quickly  and  rapidly  communicate  ideas  to  the  reader. 
Examples  of  the  use  of  mn-around,  wrap-around,  and  transparent  text 
can  be  found  in  consumer  magazines,  newspapers,  and  instructional 
materials.  In  many  instances,  legibility  appears  severely  restricted  by  the 
haphazard  mixture  of  text  and  graphics. 

Desktop  publishing  computer  programs  have  individualized  the 
page  layout  process  and  have  provided  sophisticated  graphic 
enhancements  for  computer  users  who  have  no  training  in  either 
instructional  development,  page  layout,  or  graphic  design.  People  who 
use  such  computer  software  to  design  instructional  materials  are  faced 
with  a  vast  array  of  newly  developed  features  with  no  guidelines  for  their 
effective  use.  Guidelines  are  sometimes  provided  but  they  are- often 
confusing.  In  one  guide  to  basic  design,  a  designer  is  told  to  "Frame  your 
pictures  with  white  space...White  space  separates  headlines  from 
surrounding  body  copy  and  artwork,  making  headlines  easier  to  read" 
(Parker,  1987.  pp  42-43).  The  examples  show  both  run-around  and 
wrap-around  features  being  used. 

Teachers  and  trainers  who  use  computer  programs  to  design 
instructional  materials  and  integrate  visuals  in  the  text  are  asking  for 
guidelines.  How  do  these  new  desktop  publishing  enhancements  affect 
the  legibility  of  text?  What  is  the  optimal  placement  of  text  and  graphics? 
How  and  where  should  white  space  be  located  to  enhance  reading 
comprehension?  At  what  point  does  the  integration  of  the  text  and  visual 
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FIGURE  1 

Run-Around  Text 


In  terms  of  documentation  they  all  do  a  good  job,  but  PageMakens  by  far  the 
hardest  program  to  learn  to  use.  For  users  already  familiar  with  PageMakeron  the 


Mac,  there  will  be  irc^ant  recognition  —  it's  the  same  program 
with  improvements.  Forpeople  who  haven't  struggled  to  learn  it 
already,  they  have  a  long  leaming  time  ahead  of  them.  o^i^^  Spell- 
d/nder,  too,  requires  a  long  learning  period.  It's  not  as  ^^[\  compli- 
cated as  PageMaker,  but  the  dot-command  structure  ^  makes  it 
less  intuitive  than  a  program  that  uses  a  mouse  and  ^  icons. 
With  Spe/to/nc/er,  I  had  to  constantly  referto  the  manual  \  to  look 
up  commands,  whereas  with  Pagemaker,  I  had  diffi-  cultyjust 
going  through  the  complex  menu  choices.  Ventura  wasrela- 
tively  easy  to  learn,  allowing  me  to  feel  in  control  of  the  program 
after  a  much  shorter  time.  In  fact,  Ventura  gave  me  almost 


as  much  pleasure  in  the  thrill  of  discovery  followed  by  quick  satisfaction  as  Clickart 
did  when  i  first  used  it.  If  I  had  used  Venturabeiore  Clickart,  I  think  I  would  have  been 
disappointed  with  Clickart. 


FIGURE  2 

Wrap- Around  Text 

In  terms  of  documentation  they  all  do  a  good  job,  but  PageMaker  ls  by  far  the 
hardest  program  to  leam  to  use.  For  users  already  familiar  with  PageMaker 
on  the  Mac,  there  will  be  instant  recognition  —  it's  the  same  C\P  r  o  g  r  a  m 
with  ir.iprovements.  For  people  who  haven't  struggled  to  ^>-;{^^6S''n  it  al- 
ready, they  have  a  long  learning  time  ahead  of  them.  Spell-^^^ binder,  too,  re- 
quires a  long  learning  period.  It's  not  as  complicated  as  //^>  PageMaker, 
but  the  dot-command  structure  makes  it  less  intuitive  than^^a  program  that  uses  a 
mouse  and  icons.  With  Spellbinder,  I  had  to  constantly  refer  V  to  the  manual  to 
look  up  commands,  whereas  with  Pagemaker,  I  had  difficulty  Xiust  going 
through  the  complex  menu  choices.  Ventura  was  relatively^^J  easy  to  learn, 
allowing  me  to  feel  in  control  of  the  program  after  a  much  shorter^^  time.  Intact, 
Ventura  gave  me  almost  as  much  pleasure  in  the  thrill  of  discovery  ^■followed  by 
quick  satisfaction  as  Clickart  did  when  I  first  used  it.  If  I  had  used  Ventura  before 
Clickart,  I  think  I  would  have  been  disappointed  with  Clickart. 
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FIGURE  3 

Transparent  Text 


In  terms  of  documentation  they  all  do  a  good  job,  but  PageMal<er  \s  by  far  the 
hardest  program  to  learn  to  use.  For  users  already  familiar  with  RagH^aker  on  the 
Mac,  there  will  be  instant  recognition  —  it's  the  same  progra^^ttmiprovements. 
For  people  who  haven*t  struggled  to  learn  it  already,  they  hav^a^ng  learning  time 
ahead  of  them.  Spellbinder,  too,  requires  a  long  learnif^g  period.  It's  not  as 
complicated  as  PageMaker,  but  the  dot-command  structuiWnakes  it  less  intuitive 
than  a  program  that  uses  a  mouse  and  icons.  With  Spollbino^  I  had  to  constantly 
refer  to  the  manual  to  look  up  commands,  whereas  with  PagSwratPr,  I  had  difficulty 
just  going  through  the  complex  menu  choices.  Venturav^RS  relat^ve^y  easy  to  learn, 
allowing  me  to  feel  in  control  of  the  program  after  a  much  short^rNkrie.  In  fact, 
Ventura  gave  me  almost  as  much  pleasure  in  the  thrill  of  discovery  followed  by  quick 
satisfaction  as  Clickartdld  when  I  first  used  it.  If  I  had  used  l/en/ura  before  Clickart, 
I  think  I  would  have  been  disappointed  with  Clickart. 
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adversely  affect  legibility?  How  close  to  the  visual  should  the  text  be 
placed? 

PURPOSE  OF  THE  STUDY 

The  purpose  of  this  study  was  to  determine  the  effects  of  three 
electronic  text  variables,  run-around,  wrap-around,  and  transparent  text, 
on  reading  speed  and  comprehension.  Results  mights  suggest  sor.e 
guidelines  for  the  placement  of  graphics  within  electronic  text. 

Many  researchers  have  investigated  layout,  typographic  cueing, 
and  presentation  of  graphic  materials  in  instructional  text.  For  example. 
Hartley  (1985)  determined  that  page  size  affects  other  layout  decisions. 
Recent  studies  comparing  the  effects  of  justified  and  unjustified  text  are 
inconclusive.  For  example,  Jonassen  (1982)  reported  that  in  some 
circumstances,  right  justified  text  inhibited  comprehension  when 
compared  to  unjustified  text.  Yet,  Campbell  et  al.  (1981)  and  Muncer  et 
al.  (1986)  determined  that  reading  speed  and  comprehension  were  not 
significantly  influenced  by  justification  versus  nonjustification.  Jandreau 
et  al.  (1986)  investigated  the  effects  of  varying  spaces  between  phrases 
in  sentences.  And,  others  have  examined  certain  design  features 
common  to  printed  and  electronic  text  in  the  presentation  of  graphic 
materials  (Alesandrini,  1987;  Poggenpohl,1985;  Winn, 1987;  Winn  & 
Holliday,  1982).  Graphics  designers  emphasize  the  importance  of  using 
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white  space  for  balancing  text,  positioning  graphics  as  the  dominant 
visual  element,  and  improving  the  aesthetics  of  the  page  layout  (Parker, 
1987).  And,  in  1987,  Hartley,  Isaacs,  and  others  noted  a  need  for 
published  research  on  the  effect  of  various  text-wrap  enhancements  with 
either  printed  or  electronic  text.  There  is  little  reported  research  on 
graphic  placement  in  instructional  text  and  none  that  has  produced 
guidelines  for  instructional  use  of  graphics  with  the  emerging  computer 
software  programs. 

Although  certain  features  of  electronic  text  can  be  transferred  to 
printed  text,  there  are  many  differences  between  the  two  which  suggest 
that  separate  studies  be  conducted.  The  size  of  the  screen  or  page,  the 
type  size,  and  the  ability  to  scroll  are  but  a  few  of  the  differences  which 
affect  print  and  electronic  text  comparison  studies.  Because  of  these 
differences,  a  programmatic  series  of  investigations  is  needed  to 
examine  the  effects  of  these  recent  technological  enhancements  in  each 
of  the  two  media  formats. 

Knupfer  and  Mclsaac  (1988)  responded  to  that  need  by  studying 
the  effect  of  run-around  and  wrap-around  text,  within  one  and  two 
column  formats,  on  reading  speed  and  comprehension.  That  study  found 
that  the  number  of  columns  had  no  significant  effect,  however  the  text 
wrap  did  make  a  significant  difference.  Reading  speed  was  faster  and 
comprehension  was  better  with  wrap-around  text,  as  shown  in  Figure  2, 
than  with  run-around  text,  as  shown  in  Figure  1.  In  a  follow-up  study, 
Knupfer  and  Mclsaac  (1 989)  tried  to  determine  if  that  difference  was  due 
to  the  amount  of  white  space  between  the  graphic  and  the  text,  or  due  to 
the  shape  of  the  white  space  surrounding  the  graphic.  The  follow-up 
study  tested  the  variable  of  white  space  between  the  graphic  and  t^^xt  by 
comparing  two  sets  of  reading  materials  which  both  used  wrap-around 
text  only;  the  difference  was  that  one  wrapped  the  text  tightly  against  the 
graphic,  as  shown  in  Figure  2,  while  the  other  conformed  the  text  to  the 
shape  of  the  graphic  but  left  approximately  one  quarter  inch  of  white 
space  between  the  graphic  and  the  text,  as  shown  in  Figure  4.  There 
were  no  significant  differences  between  wrap-around  text  with  no  white 
space  and  wrap-around  text  with  one  quarter  inch  of  white  space. 

This  study  continues  the  investigation  of  text-wrap  variables  by 
adding  an  additional  text-wrap  feature,  transparent  text.  If  the  results 
prove  similar,  guidelines  should  then  be  established  to  assist  teachers 
and  trainers  in  the  appropriate  use  of  this  new  type  of  desktop  publishing 
feature.   It  is  hypothesized  that  the  previous  results  will  be  replicated  and 
further,  that  the  transparent  text  will  show  greater  interference  with 
reading  speed  and  comprehension. 

Recent  work  hy  Morrison,  Ross  and  O'Dell  (1988)  examined 
attnDutes  of  computer  based  instruction,  and  attempted  to  affect  certain 
attributes  by  regulating  text  density  for  print  and  screen  display  lessons. 
Results  suggested  that  print  had  more  advantages  than  the  computer 
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FIGURE  4 

Wrap-Around  Text  With  One  Quarter  Inch  White  Space 


In  terms  of  documentation  they  all  do  a  good  job,  but  Page- 
the  hardest  program  to  learn  to  use.  For  users  already  tamil- 


PageMaker on  the  Mac,  there  will  be  instant  recognition  — 
same  program  with  improvements.  Forpeoplev^/ho  haven't 
to  learn  it  already,  they  have  a  long  learning  time  ahead 
SpellbinderAoo,  requires  a  long  learning  period.  It's  not 
cated  as  PageMaker,  but  the  dot-command  structure 
tuitive  than  a  program  that  uses  a  mouse  and  icons. 
binder,  I  had  to  constantly  refer  to  the  manual  to  look  up 
whereas  with  Pagemaker,  I  had  difficulty  just  going  through 
plex  menu  choices,  l/enfu/c?  was  relatively  easy  to  learn,  al- 
me  to  feol  in  control  of  the  program  after  a  much  shorter  time 
Ventura  gave  me  almost  as  much  pleasure  in  the  thrill  of  discovery  followed  by  quick 
satisfaction  as  Clickart6\6  when  I  first  used  it.  If  I  had  used  Ventura  before  Clickart, 
I  think  I  would  have  been  disappoiiitsd  with  Clickart. 
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makes  it  less 
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screen  in  achievement,  completion  time,  and  learning  efficiency  for  a 
statistics  lesson.  Therefore,  the  present  study  is  limited  to  the 
examination  of  print-based,  computer-generated  text . 

METHOD 

Subjects. 

One  hundred,  thirty-two  undergraduate  students  from  an 
introductopy  computer  literacy  course  at  a  major  university  in  the 
Southwest,  participated  in  the  study.  The  elective  course  was  a  400  level 
course  which  was  offered  to  meet  a  General  Studies  numeracy 
requirement.  Students  were  sophomores,  juniors,  and  seniors  who  had 
little  or  no  previous  exposure  to  computers.  The  study  was  conducted  at 
the  beginning  of  the  semester  and  all  students  in  the  course  participated. 
Students  had  little  or  no  prior  knowledge  about  the  topic  presented  in  the 
study  materials. 

Materials. 

Readings.  Readings  included  approximately  six  pages  of 
informative  text  about  hypercard  and  desktop  publishing.  These 
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materials  were  chosen  because  the  topic  was  relevant  to  the  subjects 
and  because  the  reading  level  was  geared  for  the  general  public  as 
opposed  to  teciinical  readers.  The  topic  was  suited  to  a  novice  audience 
with  no  prior  knowledge  about  either  hypercard  or  desktop  publishing. 
The  text  was  analyzed  using  the  Flesch-Kincaid  Readability  formula  and 
was  found  suita'ole  for  college  students  at  the  1 1th  and  13th  grade  levels. 

Three  sets  of  reading  materials  were  used  for  this  study.  All 
included  the  same  text  and  graphics  arranged  in  single  column  format, 
but  each  set  utilized  a  different  text  wrap  respecting  the  graphic.  The 
three  sets  included  one  with  straight-framed  graphics  (see  Figure  1,  Run- 
Around  Text),  one  with  text  tightly  fitted  around  the  graphics  (see  Figure 
2,  Wrap-Around  Text),  and  one  in  which  the  graphic  was  placed  directly 
over  the  text  (see  Figure  3,  Transparent  Text). 

Although  the  graphics  placement  within  these  three  sets  of 
reading  materials  might  not  follow  all  of  the  elements  of  good  design,  it  is 
important  to  know  that  the  way  in  which  they  were  developed  is  indicative 
of  what  practitioners  will  design.  A  sixth  grade  teacher  who  had  a 
masters  degree  in  the  design  and  utilization  of  educational  media,  and 
who  helped  to  instruct  a  desktop  publishing  course,  was  very  much 
involved  in  the  design  of  these  reading  materials.  Therefore,  any  flaws  in 
the  materials  design  are  potential  problems  that  other  teachers  and 
trainers  might  exhibit  as  well. 

Pretest  and  Posttest.  To  measure  the  students' 
comprehension  of  the  reading  material,  a  pretest  and  a  posttest  were 
developed  from  key  concepts  which  the  authors  extracted  from  the 
readings.  No  test  items  were  dependent  upon  the  information  contained 
in  the  graphics.  The  tests  were  developed  and  validated  during  a 
previous  phase  of  the  project,  and  were  found  to  be  reliable  at  the  .76 
level.  The  pretest  and  posttest  were  identical  with  one  exception;  the 
posttest  contained  an  additional  six-question  attitude  section.  The 
attitude  section  included  questions  regarding  the  students'  judgement  of 
the  difficulty  of  content,  the  degree  to  which  graphics  interfered  with  their 
comprehension  and  reading  speed,  their  degree  of  interest  in  the 
subject,  and  their  overall  understanding  of  the  matenal  after  reading  it. 
These  attitude  questions  were  measured  on  a  Likert-type  scale  and  were 
not  included  in  the  posttest  comprehension  score. 

Procedure. 

All  subjects  were  given  a  pretest  to  determine  their  knowledge  of 
the  subject  prior  to  reading.  There  was  no  time  limit  and  students  were 
free  to  take  the  time  they  needed  to  complete  the  test.  Next,  the  three 
sets  of  reading  materials  were  randomly  distributed  to  the  students  so 
that  each  student  received  one  set  of  articles.  Students  were  instructed 
to  read  through  the  text  one  time  without  hurrying.  When  subjects  were 
finished  reading,  each  of  their  reading  times  was  recorded.  Immediately 
following  the  reading,  the  subjects  were  given  the  posttest  for 
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comprehension  They  were  allowed  as  much  time  as  they  needed  to 
complete  the  test. 

RESULTS  AND  CONCLUSIONS 

The  design  of  the  study  was  an  analysis  of  variance  The 
independent  variable  was  the  text-graphic  format  (run-around,  wrap- 
around, or  transparent  text)  and  the  dependent  measures  included 
reading  speed  and  gain  in  comprehensior.  as  measured  by  the 
difference  between  the  pretest  and  posttest  scores.  The  alpha  level  for 
significance  was  set  at  .05. 

Table  1  shows  the  mean  scores  for  the  pretest,  posttest,  gam,  and  time 
elapsed  for  each  of  the  treatment  groups.  Although  the  ANOVA  showed  no 
significant  difference  in  either  comprehension  or  speed  among  the  three 
treatment  groups,  the  table  does  show  some  patterns  that  may  be  worth 
noting.  First,  \l  app-^ars  that  the  wrap-around  treatment  group  had  a  higher 
mean  gam  than  the  transparent  treatment  group,  which  in  turn  had  a  higher 
mean  gain  than  the  run-around  treatment  group.  Second,  it  appears  that  the 
wrap-around  treatment  group  also  had  the  fastest  reading  speed,  followed 
by  the  run-around  group  and  then  the  transparent    oup.  We  can  conclude 
that  the  wrap-around  treatment  group  had  the  best  comprehension  and  the 
fastest  reading  time,  even  tnough  those  results  are  not  outstanding  enough 
to  be  statistically  significant. 

TABLE  1 

Mean  Scores 


Pretest 

Posttest 

09in 

Time 

Run-Around 

13.85 

45.47 

31,62 

13  74 

Wrap-Around 

13.55 

51.91 

38.36 

12.59 

Transparent 

12.57 

47.76 

35.19 

14.51 

We  might  consider  the  possibility  of  interference  in  the  scores  of 
the  transparent  text  group.  Although  the  three  text-wrap  forms  were 
randomly  distributed,  the  transparent  group  started  off  with  a  mean 
pretest  score  that  was  a  full  point  lower  than  the  other  two  groups  There 
IS  no  explanation  for  this  occurrance.  One  might  speculate  that  the 
transparent  group  took  more  time  looking  at  the  graphics  than  the  other 
groups,  although  one  student  commented  in  a  post  study  discussion  that 
she  was  unaware  of  any  graphics  being  there  as  she  read,  suggesting 
that  she  read  right  through  the  graphics.  We  venture  to  guess  that  the 
lower  comprehension  scores  on  for  the  run-around  text  group  might  be 
because  the  text  is  more  difficult  for  the  eye  to  follow  due  one  of  three 
possible  reasons:  1 )  there  are  larger  gaps  of  white  space  that  the  eye 
must  jump  across  to  continue  reading,  2)  there  are  variations  in  the 
amount  of  white  space  that  the  eye  must  jump  across  to  continue 
reading,  or  3)  the  text  justification  might  interfere  with  reading  continuity 
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Irregular  lines  of  text  around  a  graphic  and  graphics  placed  over 
text  did  not  confuse  the  readers  in  this  study.  The  new  enhancements 
currently  available  in  desktop  publishing  software,  particularly  the  wrap- 
around features,  were  not  found  to  inhibit  reading. 

Because  a  significant  difference  in  both  reading  speed  and 
comprehension  was  found  between  the  run-around  and  wrap-around 
treatment  groups  in  a  prior  study,  we  suggest  replication  of  this  study. 
Further,  we  suggest  that  subjects  in  each  of  the  three  treatment  groups 
be  quizzed  about  their  recollection  of  the  graphics  to  see  if  they  actually 
look  at  the  graphics  as  they  read  or  if  they  concentrate  on  the  text  alone. 
Because  graphics  are  used  for  different  reasons,  such  as  decoration 
versus  information,  it  could  be  important  to  know  whether  tne  subjects 
actually  look  at  the  decorative  graphics.  It  is  possible  that  subjects' 
reading  speed  and  comprehension  with  tfie  three  text-wrap 
configurations  could  vary  dependent  upon  the  type  of  graphic  used. 

Future  studies  should  consider  perceptual  principles  such  as 
proximi'.y,  similarity,  continuity,  and  closure.  These  four  processes,  by 
which  the  mind  organizes  meaning,  depend  on  how  physically  close  the 
objects  are,  how  similar  tliey  are,  whether  there  is  a  continuous  line  to 
guide  the  eye,  and  whether  the  minimal  amount  of  information  is  present 
that  IS  necessary  to  obtain  meaning  or  closure.  Reading  speed  and 
comprehension  are  directly  affected  by  the  way  the  mind  organizes 
meaning  from  the  placement  of  graphics  and  text. 
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Overview 

This  study  is  an  investigation  of  the  effects  of 
different  loci  of  instructional  control  on  students' 
metacognition  and  cognition  in  a  computer-assisted 
learning  environment.    The  loci  of  instjnictional 
control  compared  in  this  study  are  learner  and  program 
control  over  instructional  elements  in  a  lesson.  LOGO, 
a  computer  programming  language,  is  the  content  area  to 
be  taught.    Subjects  are  third-grade  students. 

This  study  is  intended  to  provide  valuable 
guidance  for  the  design  of  educational  software. 
Addressing  conflicting  issues  regarding  instructional 
design  strategies  may  help  maximize  the  potential  of 
computers  as  educational  media. 

Need  for  Research 

As  sophisticated  vehicles  for  instruction, 
computers  can  create  highly  motivating  and  dynamic 
learning  environments  (Feurzeig,  1988) ,  and  present  a 
virtually  unlimited  range  of  options  to  instructional 
designers  and  individual  learners  (Burke,  1982;  Gay, 
1985) .    During  the  last  few  years,  educational 
technology  researchers  have  recognized  the  unique  and 
divergent  instructional  capabilities  of  computers,  and 
thus  have  responded  by  enthusiaotically  devoting 
efforts  to  the  improvement  of  computer  education.  In 
such  efforts,  there  has  been  a  great  emphasis  on 
systematic  methods  for  designing  Computer- Assisted 
Instruction  (CAI) ,  and  enhancing  the  quality  of 
educational  software. 

The  adoption  of  computers  to  educational  settings 
has  enabled  interaction  between  learners  and 
instructional  materials.    One  of  the  issues  currently 
attracting  researchers'  attention  in  the  area  of  CAI 
design  is  the  locus  of  instructional  control :  should 
instructional  control  be  given  to  a  learner  (learner 
control),  or  to  a  program  (program  control)? 

Most  early  CAI  was  designed  on  the  basis  of 
program  control  with  little  or  no  control  given  to  a 
learner  over  instructional  elements.    One  of  the 
criticisms  of  such  design  is  that  it  is  based  on  the 
assumption  that  instructional  designers  are  the  best 
judges  of  when,  where,  how  much  and  what  kind  of 
instruction  is  needed  to  teach  certain  knowledge  or 
skills  (Hannafin,  1984).     Learner  control,  in  contrast, 
is  an  instructional  design  strategy  that  provides  a 
learner  with  freedom  to  make  key  decision (s)  about 
her/his  own  learning  rather  than  being  locked  into  the 
elements  predesigned  by  an  instructor  or  instructional 
designer.     Flexibility  is  the  intended  outcome  of  this 
strategy. 
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Nunan  (1983)  contended  that  knowledge  is 
constiructed  in  the  process  of  learning,  that  a  learner 
is  an  intentional  agent,  and  that  instructional 
designers  should  therefore  seek  alternative  design 
strategies  to  reject  the  ''technological  philosophy  of 
education**  and  the  ••common-sense  structure  of 
professionalized  design**  that  have  predominated  in  the 
area  of  instructional  design.     Instead  of  program 
control,  I  propose  learner  control  as  one  of  possible 
alternatives  that  place  active  learners  at  the  center 
of  learning. 

In  sum,  in  seeking  systematic  methods  to  enhance 
the  quality  of  educational  software,  the 
nondeterministic  approach  of  learner  control  to 
instructional  design  seriously  challenges  the 
traditional  instructional  design  paradigm,  program 
control,  upon  which  most  CAI  has  been  built  (Duchastel, 
1986) .    However,  the  intuitively  appealing  assumptions 
and  theoretical  arguments  regarding  learner  control 
have  received  little  support  from  actual  research 
(Steinberg,  1977;  Carrier,  1984;  Clark,  1984) •  The 
question  of  whether  we  need  to  assign  control  to  a 
learner  or  to  a  program  has  frequently  been  asked,  but 
is  unresolved.    Therefore,  it  remains  as  an  empirical 
issue  requiring  research  attention. 

Importance  of  Learner  Control  in  Comparison  with 
Program  Control 

Advocates  of  learner  controlled  instruction  have 
discussed  its  various  advantages.    This  study  will 
focus  on  two  of  the  possible  benefits  of  learner 
control:  1)  positive  approach  to  individual 
differences,  and  2)  enhancement  of  metacognition  and 
cognition. 

One  of  the  most  important  benefits  of  learner 
control  is  its  positive  interaction  with  individual 
differences.     It  is  often  argued  that  the  unique 
advantage  of  CAI  is  its  ability  to  adapt  to  the  needs 
of  individual  students.    Theorists  and  researchers  have 
enthusiastically  attempted  to  design  individualized 
instruction  through  CAI  that  can  accommodate  various 
types  of  individuals  in  educational  practice.  However, 
while  most  people  agree  with  the  premise  that 
individual  learners  are  different  in  ways  that 
influence  performance  in  instruction,  there  seems  to  be 
little  agreement  about  the  way  in  which 
individualization  of  instruction  should  be  achieved 
(Carrier,  1984;  Gray,  1987).    On  this  point,  research 
on  learner  control  has  begun  with  the  recognition  that 
no  one  instructional  treatment  can  be  best  for  every 
learner  (Kinzie  et  al.,  1987). 

Another  important  advantage  of  learner  control  is 
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the  enhancement  of  xnetacognition  and  cognition  through 
its  dynamic  learning  processes.    Unlike  program 
controlled  instruction,  which  aims  to  improve  only 
products  of  learning,  learner  controlled  instruction 
emphasizes  processes  as  well  as  products  of  learning. 
Cognitive  psychologists  object  to  the  prescriptive 
characteristics  of  traditional  instructional  design 
under  program  control,  whereby  instruction  acts  upon  a 
learner,  as  opposed  to  a  learner  taking  responsibility 
for  her/his  own  learning  (Bonner,  1988). 

According  to  Flavell  (1979,  1987),  xnetacognition 
refers  to  the  active  monitoring  and  consequent  self- 
regulation  of  one's  own  cognitive  processes,  and 
orchestration  of  these  processes  in  the  service  of  some 
concrete  goals  or  objectives.    Under  learner  control, 
individual  learners  are  in  a  position  to  systematically 
analyze  the  nature  of  a  task;  to  monitor  and  analyze 
their  own  understanding  of  the  given  task;  to  select 
activities  thait  can  be  performed  to  accomplish  the 
task;  and  to  actively  decide  levels  or  amounts  of 
instruction  needed  to  extend  the  present  knowledge 
state,  while  continuously  monitoring  their  own 
cognitive  processes  and  performances.     By  being  aware 
of  control  options  and  deliberately  making  decisions 
about  the  use  of  the  options  under  learner  control, 
learners  can  continuously  evaluate  their  progress 
towards  a  given  objective  and  attempt  to  develop 
effective  learning  strategies  to  achieve  this 
objective.    As  a  result,  they  can  more  thoroughly 
integrate  new  learning  materials  within  their  existing 
knowledge  schema,  and  enhance  metacognitive  knowledge 
and  experiences  in  a  more  effective  way  than  they  would 
with  program  control. 

In  spite  of  the  importance  of  metacognitive 
enhancement,  instructional  designers  have  not  shown 
much  concern  for  metacognitive  development  of  learners, 
especially  as  integrated  with  subject  matter,  and  this 
may  impose  limitations  on  further  progress  in 
understanding  learning  (Bonner,  1988).  Metacognitive 
enhancement  should  be  considered  within  educational 
research  and  practice  as  a  desired  outcome  of 
instructional  interventions,  along  with  cognitive 
enhancement  such  as  achievement.    Nickerson  et  al» 
(1985)  contended  that  although  content  specific 
knowledge  is  necessary  for  the  best  performance,  it  is 
not  sufficient  to  guarantee  such  performance.     In  fact, 
as  Perkins  and  Salomon  (1989)  have  asserted,  efforts 
need  to  be  devoted  to  combining  the  teaching  of 
subject-matter  content  and  of  thinking,  as  one  of  the 
exciting  domains  of  the  next  decade  of  educational 
research  and  innovation.     Learner  control  is  one  of  the 
strategies  that  attempt  to  combine  those  two  areas. 
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following  theoretical  propositions  ar^»  made:  by 
providing  students  with  control  options  over  a  key 
instructional  element  and  by  emphasizing  processes  as 
well  as  products  of  learning,  learner  control  will 
accommodate  individual  differences  and  influence 
metacognitive  as  well  as  cognitive  enhancement  in  a 
more  effective  way  than  will  progreun  control. 


PURPOSE 

The  purpose  of  the  study  is  to  examine  the  effects 
of  two  different  loci  of  instructional  control  on 
metacognitive  and  cognitive  enhancement  and  individual 
differences,  in  a  computer-assisted  instructional 
environment.    There  are  three  major  research  goals. 
The  first  is  to  compare  the  effects  of  learner  control 
with  those  of  program  control  on  students' 
metacognition,  knowledge  acquisition  and  knowledge 
application,  with  total  subjects.    The  second  goal  is 
to  extend  the  comparison  between  learner  control  and 
program  control  by  classifying  the  total  learner 
control  group  into  two  groups,  according  to  the 
effectiveness  of  students*  uses  of  control  options 
(active  learner  control  group  and  passive  learner 
control  group) ,  and  making  comparisons  between  each  of 
the  two  learner  control  groups  and  the  program  control 
group.    The  third  goal  is  to  investigate  whether 
results  of  comparisons  between  learner  control  and 
program  control  change  in  relation  to  students* 
different  levels  of  prior  knowledge  of  mathematical 
concepts  related  to  those  in  logo  (prior  background 
knowledge) .    The  following  figures  summarize  these 
three  topics: 

1)  Topic  1  ~  with  total  learner  control  group 


XI  01    02  03 

X2  01     02  03 

XI  -  learner  control  treatment 
X2  -  program  control  treatment 

01  -  test  on  knowledge  acquisition 

02  -  test  on  knowledge  application 

03  -  interview  on  metacognition 
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2)  Topic  2  -  with  active  or  passive 
learner  control  group 


Xli 

01 

02 

03 

XI 

X12 

01 

02 

03 

X2 

0J_ 

02 

03 

XI  -  learner  control  treatment  with  total  group 
XI 1  -  learner  control  with  active  group 
XI?  -  learner  control  with  passive  group 

X2  -  program  control  treatment  with  total  group 

01  -  t:est  on  knowledge  acquisition 

02  -  test  on  knowledge  application 

03  -  interview  on  metacognition 

3)  Topic  3-  with  high  or  low  prior 

background  knowledge  group 


XI 

01 

02 

03 

A 

X2 

01 

02 

03 

XI 

01 

02 

03 

B 

X2 

01 

02 

03 

A  -  high  prior  background  knowledge  group 
B  -  low  prior  background  knowledge  group 

XI  ~  learner  control  treatment 
X2  -  program  control  treatment 

01  -  test  on  knowledge  acquisition 

02  -  test  on  knowledge  application 

03  -  interview  on  metacognition 


METHOD 

Subjects 

The  study  was  conducted  with  a  class  of  24  third 
graders  enrolled  in  an  urban  elementary  school  in 
Wisconsin  during  the  fall  of  1989*    None  of  the 
students  had  previously  learned  about  logo» 

Materials 

Students  in  both  learner  and  program  control 
treatment  groups  were  introduced  to  basic  logo  commands 
with  definitions,  rules,  examples  and  practice.  In 
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practice,  students  were  challenged  to  hit  a  given 
target  or  aim  towards  it  by  changing  the  turtle's 
position  or  heading  with  the  learned  conunand.  Two 
parallel  versions  of  programs  were  used:  one  for  the 
learner  control  group  and  the  other  for  the  program 
control  group.     Both  versions  were  delivered  by  Apple 
lie  computers  along  with  the  posttests.    The  resulting 
instructional  units  for  both  learner  control  and 
program  control  underwent  expert  review. 

Throughout  the  instruction,  students  were  able  to: 
1)  identify  the  screen  turtle  and  the  properties  of  the 
turtle,  i.e.  position  and  heading;  2)  recall  and  use 
the  logo  commands  for  moves  that  change  the  position  of 
the  turtle  in  an  immediate  mode,  i.e.  FORWARD  (FD)  and 
BACK  (BK) ;  3)  recall  and  use  the  logo  commands  for 
turns  that  change  the  heading  of  the  turtle  in  an 
immediate  mode,  i.e.  LEFT  (LT)  and  RIGHT  (RT) ;  4) 
predict  outcomes  of  commands  and  debug  commands;  and  5) 
(under  learner  control)  analyze  the  following:  how  ^ell 
do  I  understand  the  given  command?  What  don't  I  know 
about  it?  What  do  I  need  to  review  to  change  the 
present  state  of  my  knowledge? 

Treatment 

Two  types  of  treatments  were  employed  in  this 
study  on  the  basis  of  each  locus  of  instructional 
control:  learner  control  and  program  control.  The 
former  consisted  of  internally  controlled  CAI  in  which 
students  were  free  to  decide  whether  and  how  often  they 
needed  to  review  some  of  the  instructional  components 
relevant  to  a  practice  item,  or  to  skip  the  review.  In 
contrast,  the  latter  consisted  of  externally  controlled 
CAI:  all  control  in  the  lessons  was  regulated  by  the 
program  on  the  basis  of  predesigned  rules,  so  that 
students  had  no  control  over  instructional  components 
except  the  instructional  pace  —  they  could  advance 
through  the  instruction  when  they  were  ready. 

program  control:  In  the  tutorial  portion  of  each 
lesson,  students  learned  the  definition,  the  rule  and 
four  examples  of  a  logo  command.    Then,  four  practice 
items  were  presented.    When  a  correct  response  was 
made,  students  received  positive  feedback  and  moved  to 
the  next  practice  item.    When  an  incorrect  response  was 
made,  the  program  provided  students  with  negative 
feedback  and  automatically  branched  them  to  the  review 
of  the  definition,  the  rule  and  an  example  of  the 
command  under  consideration.     The  review  of  instruction 
for  practice  was  presented  only  after  the  first  wrong 
answer.    Under  program  control,  when  students  exhibited 
a  lack  of  understanding  of  a  given  logo  command,  they 
were  guided  to  review  instructional  elements  relevant 
to  the  command.     Such  repetitive  instruction  may 
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reinforce  students*  understanding  of  a  given  topic. 

After  the  review,  students  had  a  second 
opportunity  to  respond  to  the  practice  item*     If  "che 
answer  was  still  wrong,  personalized  feedback  was 
presented  to  show  the  correct  answer,  and  the  next 
practice  item  was  presented.    At  the  end  of  the 
practice  session,  a  total  score  was  displayed. 

l^^m^r  control:  The  differences  between  learner 
control  and  program  control  were  made  only  in  the 
presentation  of  the  question  for  self-estimates  of 
understanding  on  a  given  command  and  in  the  way  in 
which  review  was  presented  after  a  wrong  answer.  After 
the  tutorial  portion  of  a  lesson,  as  a  part  of  the 
treatment,  students  were  asked  to  rate  their 
understanding  of  the  given  command  on  a  four-item 
Likert  scale  (1  for  "not  at  all"  and  4  for  "very 
well"),  and  to  solve  four  practice  items.    When  an 
incorrect  response  was  given  to  a  practice  item,  they 
were  told  that  their  answer  was  wrong  and  were 
continuously  advised  to  think  about  what  they  did  not 
clearly  understand  about  the  command  learned  and  what 
they  should  review.    Then,  students  were  allowed  to 
review  the  instruction  for  practice,  the  definition  and 
the  rule  of  the  command,  or  an  example  of  the  command, 
as  many  times  as  they  wanted  to,  or  to  skip  the  review. 
Every  time  they  used  a  review  option,  a  summary  of 
their  performance  in  practice  and  uses  of  control 
options  was  provided.    When  students  decided  not  to 
review  any  more,  a  question  was  presented  to  confirm  if 
they  understood  the  command  well  enough  to  try  the 
practice  item  again.    Then,  with  confirmation,  they  had 
an  opportunity  to  solve  the  same  practice  item  a  second 
time.    When  the  practice  session  was  completed,  a  total 
score  was  reported  along  with  the  students"  self- 
estimates  of  their  understanding  of  the  given  command. 

In  addition,  the  following  factors  were  integrated 
into  the  design  of  learner  controlled  instruction,  as 
recommended  by  researchers  and  theorists.  First, 
control  options  were  clearly  labeled  to  help  students 
use  control  options  effectively  (Reigeluth  and  Stein, 
1983) .     Second,  immediate  feedback,  continuous  advice 
on  students"  progress  and  summaries  of  their  uses  of 
control  options  were  presented  to  help  students  make 
informed  decisions  about  their  own  learning  (Hannafin, 
1984;  Kinzie,  1988;  Lepper,  1985;  Lepper  &  Malone, 
1987;  Markman,  1977;  Merrill,  1984;  Tennyson,  1980, 
1981;  Tennyson  &  Buttrey,  1980;  Waldrop,   1984).  Third, 
students  were  required  to  solve  a  certain  number  of 
practice  items  to  assure  that  they  did  not  bypass 
important  instructional  components  (Kinzie,  1988; 
Steinberg,  1977;  Waldrop,  1984).     Fourth,  prior  to  the 
instruction,  there  was  a  pretraining  period  to  help 
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students  become  familiar  with  the  novel  learning  system 
with  contrri  options,  perform  conscious  cognitive 
information  processing,  and  accjuire  declarative 
(knowing  what) ,  procedural  (knowing  how)  and 
conditional  (knowing  why  and  when)  knowledge  involved 
in  building  their  own  learning  strategies  (Campbell, 
1964;  Gay,  1986;  Hannafin,  1984;  Holmes  et  al.,  1985; 
Kinzie».  1988;  Merrill,  1975,  1984;  Schimmitt  &  Newby, 
1985;  Steinberg,  1977;  wydra,  1980). 

Procedure 

Prior  to  the  experiment,  on  the  basis  of  a  mean 
score  on  a  test  on  mathematical  concepts  which  are 
specifically  related  to  logo  (e.g.  distance  and 
direction) ,  students  were  identified  as  belonging  to 
either  a  high  or  low  prior  background  knowledge  group. 
Then,  in  each  group,  students  were  randomly  assigned  to 
either  the  learner  control  or  the  program  control 
treatment  condition.    To  motivate  students,  they  were 
informed  that  although  scores  on  tests  would  not  affect 
their  class  grades,  their  teacher  would  take  a  look  at 
their  scores.     Such  information  should  be  presented  to 
stimulate  an  actual  classroom-like  incentive  that  is 
missing  in  most  learner  control  studies  (Tennyson, 
1980;  Tennyson  &  Buttrey,  1980). 

The  study  was  conducted  over  a  three-week  period 
with  three  sessions  per  week.    Each  session  took  about 
40  minutes.     Before  instruction  began,  there  was  a 
pret raining  section  aimed  at  help  students  to  become 
familiar  with  computers,  instructional  systems  and 
control  options,  and  to  understand  the  declarative, 
procedural  and  conditional  knowledge  involved  in  using 
control  options.     Then,  lessons  on  logo  commands  were 
presented.     In  each  session  of  instruction,  those  who 
finished  a  lesson  earlier  than  the  others  were  asked  to 
remain  in  the  lab  and  spend  the  rest  of  the  session 
with  a  paper-and-pencil  task  on  a  maze.  Upon 
completion  of  all  instruction,  posttests  on  knowledge 
acquisition  and  application  and  an  interview  on 
metacognition  were  conducted. 

Dependent  Variables 

1)  Interview  on  Metaoognition:  To  serve  the  major 
purpose  of  this  study,  an  adaptation  of  the 
Metacomponential  Interview  developed  by  Clements  and 
Nastasi  (1988)  vas  employed.    This  interview  instrument 
was  developed  to  assess  children's  specific  executive 
levels  of  metacognitive  functioning  in  a  mediated 
situation  with  aid  through  a  series  of  prescribed 
prompts  for  mediational  purposes.    Although  five 
metacomponents  of  the  seven  proposed  by  Sternberg 
(1985)  are  included  in  this  instrument,  in  relation  to 
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the  learning  experience  gained  through  this  study,  only 
one  metacomponent,  monitoring  solution  processes,  was 
employed  in  assessing  metacognitive  effects. 

Four  questions  were  presented,  with  five  prompts 
per  q[uestion  (appendix  3).    The  nvimber  of  prompts 
required  for  the  solution  of  each  question  was  summed 
and  transformed  so  that  a  higher  score  refers  to  fewer 
prompts  required  and  better  performance.     Further,  the 
transformed  scores  were  summed  again  over  question 
items.     In  addition,  as  Clement  and  Nastasi  directed, 
the  following  measures  were  ta)cen  for  each  question:  1) 
the  number  of  prompts  needed  to  demonstrate  the 
identification  of  the  fact  that  something  was  wrong  in 
presented  information  and  induction  (utilization) ;  and 
2)  the  number  of  prompts  needed  to  answer  correctly 
(correctness) .    The  Cronbach's  a  reliability  values 
obtained  were  .53  for  utilizaticr  and  .60  for 
correctness.     The  entire  interview  process  was  recorded 
for  later  analysis  and  used  to  test  inter-rater 
agreement  in  scoring.    The  correlation  values  for 
inter-rater  agreement  were  .87  for  utilization  and  .92 
for  correctness. 

2)  Test  on  Knowledge  Acquisition:  A  16-item 
posttest  was  administered.     In  the  test,  items  similar 
to  the  practice  ones  embedded  in  the  instruction  were 
presented  to  explore  students'  knowledge  acquisition. 
The  number  of  test  items  on  each  logo  command  was 
equally  balanced  across  all  the  commands  taught 
throughout  the  instruction.     The  Cronbach's  a 
reliability  value  among  the  test  items  for  each  command 
was  .86. 

3)  Test  :>n  Knowledge  Application:  Given  the  length 
of  each  side  of  a  square  on  the  screen,  students  were 
asked  to  draw  a  square  by  changing  the  turtle  move  and 
turn.     In  order  to  avoid  any  possible  confounding 
effects  due  to  the  use  of  multi-media,  both  tests  on 
knowledge  acquisition  and  application  were  delivered  by 
computers  along  with  instruction,  and  students  worked 
individually  with  assigned  computers. 


HYPOTHESES 

To  serve  the  purpose  of  this  study,  the  following 
hypotheses  were  examined: 

1)  The  learner  control  group  will  phow 
significantly  higher  scores  than  the  program  control 
group  in  the  tests  on  metacognition,  knowledge 
acquisition  and  knowledge  application. 

2)  Significant  differences  between  learner  control 
and  program  control  will  be  more  apparent  when  the 
active  learner  control  group  is  compared  with  the 
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program  control  group  than  when  the  passive  learner 
control  group  is  compared  with  the  program  control 
group . 

3)  For  both  the  group  with  high  prior  background 
knowledge  and  the  group  with  low  prior  background 
knowledge,  the  learner  control  group  will  demonstrate 
higher  scores  than  the  program  control  group  in  the 
tests  on  metacognition,  knowledge  acquisition  and 
knowledge  application. 


DATA  AMALYSxS  AND  RESULTS 

Analysis  of  Variance  (ANOVA)  methods  were  utilized 
in  examining  the  effects  of  two  different  loci  of 
instructional  control  on  metacognition  (monitoring 
solution  processes) ,  knowledge  acquisition  and 
knowledge  application. 

Total  Learner  Control  vs.  Program  Control 

First,  the  performance  of  all  students  under 
learner  control  in  the  interview  on  metacognition 
(utilization  and  correctness  of  monitoring)  and  the 
tests  on  knowledge  acquisition  and  application  was 
compared  with  that  of  all  students  under  program 
control,  on  the  basis  of  an  ANOVA.  Significant 
differences  were  found  in  both  utilization  of 
monitoring  and  correctness  of  monitoring  (Table  3). 
Regarding  the  metacognitive  effects  due  to  the 
treatment,  students  under  learner  control  showed 
significantly  high^ir  '•;>oores  in  both  the  utilization  and 
correctness  of  monitvoring  than  students  under  program 
control.    On  knowledge  acquisition  and  application, 
students  urder  learner  control  showed  higher  scc::es 
than  those  under  program  coni:rol,  but  no  significant 
difference  between  the  two  treatments  was  tound. 


Insert  Table  3,  Figure  2.1  and  Figure  3.2  about  here 


Active  or  Passive  Learner  Control  vs.  Program  Control 

Then,  to  permit  further  investigation  of  the 
comparative  effects  of  the  two  treatment  conditions, 
students  under  learner  control  weio  classified  into  two 
groups  according  to  the  effectiveness  of  their  usage  of 
control  options.    An  AlIOVA  was  conducted  to  compare 
each  of  the  two  learner  control  groups  (active  and 
passive)  with  the  total  program  control  group. 
Effectiveness  was  evaluated  oi.  the  basis  of  the 
accuracy  of  the  self-estimates  of  understanding  of  the 
logo  command  taught  and  the  appropriateness  of  the  uses 
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of  control  options;  the  relatic  .  between  one's  rate  of 
understanding  on  a  four- item  Likert  scale  and  her/his 
subsequent  performance  in  practice,  and  the  relation 
between  the  number  of  reviews  and  the  number  of  wrong 
answers  in  practice  (appendix  2) . 

First,  to  compare  the  active  loarner  control  group 
with  the  program  control  group,  an  ANOVA  was  conducted 
on  the  four  dependent  variables  (table  4) •     In  contrast 
with  the  results  of  the  comparison  between  the  total 
learner  control  and  the  program  control  treatment 
groups,  a  significant  difference  was  found  in  knowledge 
acquisition  and  more  significant  differences  were  found 
in  utilization  and  correctness  of  monitoring.  However, 
no  significant  difference  was  found  in  knowledge 
application. 


Insert  Table  4 ,  Figure  4 . 1  and  Figure.  4 . 2  about  here 


Second,  to  compare  the  passive  learner  control 
group  with  the  program  control  group,  an  ANOVA  method 
was  also  utilized  with  the  four  dependent  variables 
(table  5) .    No  significant  difference  was  found  in  any 
of  the  dependent  variables  between  the  two  treatment 
groups . 


Insert  Table  5,  Figure  5.1  and  Figure  5.2  about  here 


Learner  Control  vs>  Program  Control: 

With  High  or  Low  Prior  Background  Knowledge  Groups 

In  addition,  in  order  to  explore  the  effects  of 
two  different  loci  of  instructional  control  on 
individual  differences  (different  levels  of  prior 
background  knowledge) ,  students  were  classified  into 
two  groups  (high  or  low  prior  background  knowledge 
groups)  on  the  basis  of  a  test  assessing  knowledge  of 
mathematical  concepts  related  to  logo.     For  each  of  the 
two  groups,  an  ANOVA  was  conducted  to  compare  the 
effects  of  learner  control  on  metacognition,  knowledge 
acquisition  and  knowledge  application  with  those  of 
program  control.    Further,  an  ANOVA  method  was  also 
used  to  compare  the  high  prior  background  knowledge 
group  with  the  low  prior  background  knowledge  group  on 
the  accuracy  of  students'  self -estimates  of 
understanding  of  a  given  logo  command,  appropriateness 
of  the  uses  of  control  options  and  effectiveness  of  the 
uses  of  control  options. 

First,  with  the  students  who  demonstrated  high 
prior  knowledge  of  mathematical  concepts  related  to 
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logo,  the  comparison  between  the  learner  control 
treatment  and  the  program  control  treatment  was  made  by 
conducting  an  ANOVA  on  the  dependent  variables  (table 
6) •    Significant  differences  between  the  two  treatment 
groups  were  identified,  in  favor  of  learner  control,  in 
all  the  four  dependent  variables:  utilization  of 
monitoring,  correctness  of  monitoring,  knowledge 
acquisition  and  knowledge  application. 


Insert  Table  6,  Figure  6.1  and  Figure  6.2  about  here 


Second,  with  those  who  demon:strated  low  levels  of 
prior  background  knowledge,  the  comparison  between  the 
l-^.arner  control  and  the  program  control  treatments  was 
made  on  the  basis  of  an  ANOVA  on  the  dependent 
variables  (table  7) .    Although  the  overall  performance 
of  the  learner  control  group  was  better  than  that  of 
the  program  control  group,  no  significant  difference 
was  found  in  any  of  the  variables  between  the  two 
treatment  groups. 


Insert  Table  7,  Figure  7.1  and  Figure  7.2  about  here 


Additionally,  an  ANOVA  on  the  accuracy  of  self- 
estimates  of  understanding ^  the  appropriateness  of  the 
uses  of  control  options  and  the  effectiveness  of  the 
uses  of  control  options  was  conducted  to  investigate 
any  differences  between  the  high  and  low  ^rior 
background  knowledge  groups  (table  8) .  Significant 
differences  were  found  in  all  those  variables  between 
the  two  groups,  in  favor  of  the  group  with  high  prior 
background  knowledge. 


Insert  Table  8,  Figure  8.1  and  Figure  8.2  about  here 


DISCUSSION 

In  this  study,  pretraining  was  provided  to  both 
treatment  groups  in  order  to  help  students  become 
favailiar  with  the  given  learning  systems.     Supported  by 
this  pretraining  along  with  the  presentation  of  clearly 
labeled  control  options  and  a  basic  requirement  to 
solve  a  certain  number  of  practice  items,  students 
under  learner  control  were  asked  to  estimate  their  own 
understanding  of  the  given  logo  command  and  make 
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"informed  decisions"  in  using  learner  control  on  the 
basis  of  immediate  feedback,  continuous  advice  and 
summaries  of  their  performance  and  uses  of  control 
options • 

When  these  supportive  factors  were  integrated  into 
the  design  of  a  learner  control  study,  learner  control 
seemed  to  foster  students'  metacognitive  as  well  as 
cognitive  abilities  in  a  more  effective  way  than  would 
program  control,  regardless  of  different  levels  of 
prior  background  knowledge  of  a  related  subject.  The 
overall  performance  of  the  rotal  students  under  learner 
control  in  monitoring  solution  processes  and  in 
recalling  and  applying  leatTied  knowledge  was  higher 
than  that  under  program  control.    In  particular, 
significant  differences  v/ere  found  in  the  assessment  of 
metacognition  in  favor  of  learner  control.  Under 
learner  control,  one  becomes  responsible  for  her/his 
own  learning,  practices  conscious  reflection  on  her/his 
cognitive  abilities,,  task  demands  and  learning 
strategies,  and  learns  how  to  manage  her/his  own 
thinking  processes  and  learning  activities.    Thus,  one 
can  be  led  to  enhance  her/his  metacognitive  abilities 
to  perform  and  behave  strategically^    These  learned 
strategies  may  be  those  that  transfer  to  other  contexts 
and  facilitate  learning  activities  effectively  and 
efficiently. 

Unlike  the  effects  on  metacognition,  the 
difference  between  the  learner  and  program  control 
treatments  in  the  effects  on  cognition  (knowledge 
acquisition  and  application)  was  not  statistically 
significant.    However,  when  an  extended  comparison 
between  the  two  treatments  was  made  with  the  active 
learner  control  group  that  exhibited  effective  uses  of 
control  options,  a  significant  difference  was  found  in 
knowledge  acquisition  in  favor  of  learner  control,  it 
seems  that  only  when  students  under  learner  control  can 
use  given  control  options  effectively  by  making 
accurate  self -estimates  of  understanding  and 
appropriate  uses  of  control,  they  can  obtain 
significantly  more  benefit  in  enhancing  cognition  as 
well  as  metacognition  than  students  under  program 
control.    Although  in  this  study  students  under  learner 
control  demonstrated  better  performance  in  the 
assessment  of  cognition  with  aids  from  supportive 
factors  built  into  the  design  of  instruction,  they 
still  showed  different  levels  of  effectiveness  in  using 
control  options.    Further  investigation  is  needed  of 
such  supportive  factors  that  can  help  students  use 
control  effectively. 

In  addition,  concerning  students'  prior  knowledge 
of  mathematical  concepts  related  to  those  in  logo,  in 
the  learner  control  treatment  group,  students  who 
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exhibited  relatively  high  levels  of  prior  background 
knowledge  were  significantly  more  capable  of  using  the 
given  control  options  than  were  those  with  low  prior 
background  knowledge.    Students  with  low  prior 
background  knowledge  tended  to  estimate  their  own 
understanding  inaccurately  and  to  review  too  little. 
Further,  among  the  students  with  high  prior  background 
knowledge,  different  effects  between  learner  control 
and  program  control  were  significantly  apparent  in 
favor  of  learner  control,  in  the  assessment  of 
metacognition  and  cognition.    However,  only  slight 
differences  between  the  two  treatments  were  found  among 
the  students  with  low  prior  background  knowledge,  in 
favor  of  learner  control. 

These  findings  about  individual  differences  in 
prior  background  knowledge  indicate  that  students' 
prior  knowledge  of  a  subject  matter  relevant  to  the  one 
learned  is  important  in  helping  them  to  understand  the 
underlying  strxicture  of  a  given  instruction,  to  develop 
effective  learning  strategies  and  thus  to  enhance 
metacognitive  and  cognitive  abilities.    As  Snow  (1980) 
contended,  learner  controlled  instruction  may  not 
overcome  the  fact  that  "individual  characteristics  not 
under  control  of  the  individual  will  determine  to  a 
significant  extent  what  and  how  much  that  individual 
will  learn  in  a  given  instructional  setting"  (p.l). 
Snow  argued  that  research  on  learner  control  generally 
has  not  proved  optimal  performance  because  it  is  not 
known  which  type&  of  learners  should  be  granted  control 
and  under  which  conditions. 

In  this  study,  in  attempting  to  find  ways  to 
overcome  problems  that  students  with  low  prior 
background  knowledge  may  have  under  learner  control, 
supportive  factors  (pretraining,  provision  of  clearly 
labeled  options  a^id  basic  requirements,  and 
presentation  of  feedback;  advice  and  summaries)  were 
integrated  into  the  c^esign  of  instruction.    With  these 
supports,  students  under  learner  control  exhibited 
better  performance  than  that  of  those  under  program 
control  regardless  of  thair  different  levels  of  prior 
background  knowledge.    This  finding  is  contradictory  to 
that  of  some  previous  research  on  learner  control 
(Goetzfried  and  Hannafin,  1985;  Hannafin,  1984;  Kinzie, 
1988;  Krendle  and  Lieberman,  1988;  Steinberg,  1977; 
Tennyson,  1981)  which  supported  the  Achievement- 
Treatment  Interaction  theory  (Tobias,  1976) ;  for 
students  with  a  low  level  of  prior  knowledge,  learner 
control  is  less  effective  than  program  control  and  for 
those  with  a  high  level  of  prior  knowledge,  learner 
control  is  more  effective  than  (or  comparable  to) 
program  control. 

It  seems  that  learner  control  can  convey  the  role 
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of  providing  opportunities  to  improve  metacognitive 
monitoring  abilities,  while  attempting  to  teach 
knowledge  of  specific  subject  matter  content  in  a  more 
effective  way  than  can  prograua  control,  regardless  of 
learners'  different  levels  of  prior  knowledge  of  a 
relevant  subject  matter.    However,  it  should  be  noted 
that  this  study  has  been  conducted  with  a  relatively 
limited  application  of  control  in  introductory 
instruction,  and  for  a  short  time  period.  Considering 
the  potentially  rich  and  unique  learning  environment 
and  experiences  that  learner  control  can  provide,  more 
research  efforts  should  be  devoted  to  developing 
instructional  models  of  learner  control  from  which  all 
students  can  benefit  in  enhancing  metacognitive  as  well 
as  cognitive  abilities. 
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APPENDIX  1 


Total  Possible  Scores  for  Metacoqnition, 
Knowledge  Accmisition  and  KnovXedae  Application 


variables 


total  possible  score 


monitoring:  utilization  24 

monitoring:  correctness  24 

knowledge  acquisition  32 

knowledge  application  70 


Means  and  Standard  Deviations  for  Metacognition. 
Knowledge  Acquisition  and  Knowledge  Application 


variables 


total  Ic  pc         low  high 

all    active  passive 


monitoring: 

10 

.96 

12 

.42 

12 

.67 

12 

.17 

9 

.50 

10 

.33 

11 

.58 

utilization 

(3 

.SO) 

(4. 

12) 

(3. 

20) 

(5. 

19) 

(2. 

94) 

(4. 

31) 

(3. 

29) 

monitoring: 

7 

.21 

8 

.83 

10 

.33 

7 

.33 

5 

.58 

6 

.00 

8 

.42 

correctness 

(4 

.25) 

(4. 

45) 

(2. 

58) 

(5. 

61) 

(3. 

50) 

(4. 

65) 

(3. 

60) 

knowledge 
acquisition 

19 

.04 

21 

.33 

23 

.33 

19 

.33 

16 

.75 

16 

.50 

21 

.58 

(6 

.92) 

(7. 

,52) 

(7. 

69) 

(7. 

47) 

(5. 

67) 

(6. 

10) 

(7. 

00) 

knowledge 

46 

.04 

50 

.42 

55 

.00 

45 

.83 

41 

.67 

37 

.50 

54 

.58 

application 

(17 

.75) 

(17. 

,51) 

(15. 

49) 

(19. 

60) 

(17. 

62) 

(18. 

23) (12. 

87) 

Note:  Standard  Deviations  are  presented  in  parentheses. 
Ic  =  learner  control  treatment  group 

all  =  total  subjects  under  learner  control 
active  =  the  learner  control  group  that  showed 

effective  uses  of  control  options 
passive  -  the  learner  control  group  that  showed 
ineffective  uses  of  control  options 
pc  =  program  control  treatment  group 
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Table  3 

TGTKL  T.KABWRB  COMTBOL  VS.  PBOGBJUl  OOMntOLt 

1^  Means  and  Standard  Deviations  for  Metacoqnitlon. 
Knowledge  Accmisitlon  and  Knowledge  Application 


21  F  Ratios 

and  F  Probabilities  between  the  Treatment  Groups 

control 

mean 

standard 

F 

F 

condition 

deviation 

Ratio 

Prob. 

variables 

monitoring: 

12.42 

4 , 12 

3.9832 

.0585  * 

utilization 

pc 

9.50 

2.94 

monitoring: 

Ic 

8.83 

4.45 

3.9544 

.0593  * 

correctness 

pc 

5.58 

3.50 

knowledge 

Ic 

21.3 

7.52 

2.8384 

.1062 

acquisition 

pc 

16.75 

5.67 

knowledge. 

Ic 

50.42 

17.51 

1. 4885 

.2354 

application 

pc 

41.67 

17.62 

*  p< .  10 

Note:  Ic  =  learner  control  treatment  group 
pc  =  program  control  treatment  group 
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Figure  3.1 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
WITH  TOTAL  LEARNER  CONTROL 


LC     [S3  PC 


Tearner  control  vs.  program  control 

WITH  TOTAL  LEARNER  CONTROL 


LC:  utilization 
PC:  utilization 

LC:  correctness 
PC:  correctness 

LC:  acquisition 
PC:  acquisition 

LC:  application 
PC:  application 


20      30       40      60      60  70 
D  High/Low      B  Mean 
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Table  4 

JkCTIVl  LEARNER  COMTSOL  VS.  PROGRAM  OOMTROL: 

1^  Means  and  Standard  Deviations  for  Metacoanitlon. 

Knowledge  Acquisition  and  Knowledge  Application 
2)  F  Ratios  and  F  Probabilities  between  the  Treatment  Groups 


control         mean         standard  F  F 

condition  deviation         Ratio       Prob  * 


variables 


monitoring:         Ic  12.67  3-20  4.3857       .0525  * 

utilization       pc  9.50  3.94 

monitoring         Ic  10.33  2.58  8.5800      .0098  ** 

correctness        pc  5.58  3.50 

knowledge  Ic  23.33  7.69  4.2701       .0554  * 

acquisition        pc  16.75  5.67 

knowledge  Ic  55.00  15.49  2.4645  .1360 

application        pc  41.67  17.62 

*  p<.10 
**  p<.01 

Note:  Ic  =  learner  control  treatment  group 

pc  =  program  control  treatment  group 
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rigure  '4,1 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
WITH  ACTIVE  LEARNER  CONTROL 


UTILIZATION 


CORRECTNESS 


ACQUISITION 


LC 
PC 


12.67 
9.6 


10.33 
6.68 


23.33 
16.76 


LC    ESS  PC 


Figure  '4.2 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
WITH  ACTIVE  LEARNER  CONTROL 


LC:  utilization 
PC:  utilization 

LC:  correutness 
PC:  correctness 

LC:  acquisition 
PC:  acquisition 

LC:  application 
PC:  application 


10      20      30      40      60  60 
D  High/Low  Mean 
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Table  5 

PAS8IVB  LE7.iBMBR  COSTBOL  VS.  PROGKaJI  OOVTSOL: 

1^  Means  and  Standard  Deviations  for  Metacognition, 
Knowledge  Acquisition  and  Knowledge  Application 
2)  F  Ratios  and  F  Probabilities  between  the  treatment  groups 


control         mean         standard  F  F 

condition  deviation         Ratio        Prob  < 


variables 


monitoring: 

Ic 

12. 17 

5.  19 

1.9802 

.  1785 

utilization 

pc 

9.50 

2.94 

monitoring 

Ic 

7.33 

5.  61 

.6707 

.4249 

correctness 

pc 

5.58 

3.  50 

knowledge 

Ic 

19.33 

7.47 

.6741 

.4237 

acquisition 

pc 

16.75 

5.  67 

knowledge 

Ic 

43.83 

19.60 

.2082 

.6543 

application 

pc 

41.67 

17.62 

Note:  Ic  =  learner  control  treatment  group 
pc  =  program  control  treatment  group 
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Figure  5^1 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
WITH  PASSIVE  LEARNER  CONTROL 


LC  E^PC 


Fi cure  5 . 2 

LcARNER  CONTROL  VS.  PROGRAM  CONTROL 
WITH  PASSIVE  LEARNER  CONTROL 


LC:  utilization 
PC:  utilization 

LC:  correctness 
PC:  correctness 

LC:  acquisition  - 
PC:  acquisition 

LC:  application 
PC:  application 


10      20      30      40      60      60       70  80 
0  High/Low      B  Mean 
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Tabltt  6 

HIGH  F&IOR  Background  Imovlafig*  6S0UP 
LHABWItR  OOHTBOL  VS.  PBOGSAH  COHTBOLX 

1)  Means  and  Standard  Deviations  for  Metacoanitloh. 

Knowledge  Accmisition  and  Knowledge  Application 
2^  F  Ratios  and  F  Probabilities  between  the  Treatment  Groups 


control 
condition 


variable 


mean 


standard 
deviation 


F 

Ratio 


F 

Prob. 


monitoring:  Ic 
utilization  pc 


13.17 
10.00 


2.93 
3.03 


3.3865       .0956  * 


monitoring:  Ic 
correctness  pc 


10.33 

6.50 


2.58 
3.62 


4.4604        .0608  * 


knowledge  Ic 

acquisition  pc 

knowledge  Ic 

appl ication  pc 


25.50 
17.67 

60.83 
48.33 


5.32 
6.53 

4.92 
15.71 


5.1879        .0460  ** 


3.4615       .0924  * 


*  p<.10 
**  P<.05 


Note:  Ic  =  learner  control  treatment  group 
pc  =  program  control  treatment  group 
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Figure  6.1 


LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
HIGH  PRIOR  BACKGROUND  KNOWLEDGE  GROUP 


OH  LC    ISSI  PC 


Figure  6.2 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
HIGH  PRIOR  BACKGROUND  KNOWLEDGE  GROUP 


LG:  utilization 
PC:  utilization 

LC:  correctness 
PC:  correctness 

LC:  acquisition  - 
PC:  acquisition 

LC:  application 
PC:  application 
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Tabla  7 

LOW  PRIOR  Ba'T'tkgroiind  knowlttdg*  CffiOUlP 
T.«>iiinro  OOITSOL  VS.  PROGRAX  OOMTRCLt 

1^  Means  and  Standard  Deviations  for  Metacoan Ltion. 
Knowledge  Acquisition  and  Knowledge  Application 


control 

mean 

standard 

r 

r 

condition 

deviation 

Ratio 

Prob. 

variables 

monitoring : 

Ic 

11. 

67 

5.24 

1.1636 

.3060 

utilization 

pc 

9. 

00 

5.24 

monitoring : 

Ic 

7. 

33 

5.61 

.9846 

.3445 

correctness 

pc 

4. 

67 

3.44 

knowledge 

Ic 

17. 

17 

7.39 

.1321 

.7238 

acquisition 

pc 

15. 

83 

5.12 

knowledge 

Ic 

40. 

00 

19.75 

.2083 

.6478 

application 

pc 

35. 

00 

18.17 

Note:  Ic  = 

learner  control 

treatment  group 

pc  =  program  control  treatment  group 
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Figure  7.1 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
LOW  PRIOR  BACKGROUND  KNOWLEDGE  GROUP 


LC  ES3PC 


Figure  7.2 

LEARNER  CONTROL  VS.  PROGRAM  CONTROL 
LOW  PRIOR  BACKGROUND  KNOWLEDGE  GROUP 


LC:  utilization 
PC:  utilization 

LC:  correctnesd 
PC:  correctnedd 

LC:  acquisition 
PC*  acquldltlon 

LC:  application 
PC:  application 
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Tabl«  8 

HIGH  VS.  LOW  FSIOR  Baokgroimd  ]mov'«dg«  cmoUPS 

1^  Means  and  Standard  Deviations  for  AccuraGv.  Appropriateness 

and  Effectiveness  under  Learner  Control 
2)  F  Ratios  and  F  Probabilities  between  the  Groups 

control       mean       standard  F  F 

condition  deviation       Ratio  Prob. 

variables 


accuracy  high 

3.  25 

.45 

37. 0000 

low 

1.71 

.45 

appropriateness  high 

3.08 

.77 

7.8715 

low 

2.04 

.49 

effectiveness  high 

3. 17 

.58 

18.9173 

low 

1.88 

.44 

*  p<.05 
**  p<.01 
***  p<.001 

Note:  high  =  high  prior  background  knowledge  group 
low  =  low  prior  background  Icnowledge  group 
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Figure  8.1 

HiGH  VS.  LOW  PRIOR  BACKGROUND  KNOWLEDGE 
ACCURACY.  APPROPRIATENESS.  EFFECTIVENESS 


HIGH     ^  LOW 


Figure  8.2 

HIGH  VS.  LOW  PRIOR  BACKGROUND  KNOWLEDGE 
ACCURACY.  APPROPRIATENESS.  EFFECTIVENESS 


HIGH:accuracy 
LOW:accuracy 

HlQH:appropriatene88 
LOW:approprlatenes3 

HIGH-effectlveness 
LOW:effectlvene88 


1  2  3 

D  High/Low      "B  Mean 
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APPENDIX  2 


ACCnnUkCY  of  SBUr-BSTnOkTBS  of  UKOBBSnMDIXO 


If  SELF-EST="NOT  AT  ALL" 
If  SELF-ESIVNOT  AT  ALL" 
If  SELF-EST="NOT  AT  ALL" 
If  SELF-EST="NOT  AT  ALL" 
If  SELF-EST="NOT  AT  ALL" 


&  WRNO*=  7  or  8,  then  ACCU]R«=  4. 

&  WRNO»  5  or  6,  then  ACCUR=  3. 

&  WRNO»  3  or  4,  then  ACCUR«  2. 

&  WRNO»  1  or  2,  then  ACaJR=  1. 
&  WRNO*  0,  then  ACCUR=  0. 


If  SEIiF-EST="A  LITTLE"  & 
If  SELF-EST="A  LITTLE"  & 
If  SELF-EST="A  LITTLE"  & 
If  SELF-EST="A  LITTLE"  & 
If  SELF-EST="A  LITTLE"  & 


WRNO=  7  or  8,  then  ACCUR=  2. 
WRNO=  5  or  6,  then  ACCaR=  4. 
WRNO=  3  or  4,  then  ACCUR=  3. 
WRNO=  1  or  2,  then  ACCUR=  1. 
WRNO=  0,  then  ACCUR=  0. 


If  SELF-EST="WELL"  & 

If  SELF-EST="WELL"  & 

If  SELF-EST«"WELL"  & 

If  SELF-EST="WELL"  & 

If  SELF-EST="WELL"  & 


WRNO=  7  or  8,  then  ACCUR=  0. 
WRNO=  5  or  6,  then  ACCUR=  1. 
3  or  4,  then  ACCUR=  4. 


WRNO= 
WRNO= 


1  or  2,  then  ACCUR=  3. 


WKNO=  0,  then  ACCUR=  2. 


If  SELF-EST="VERY  WELL"  & 
If  SELF-EST="VERY  WELL"  & 
If  SELF-EST="VERY  WELL"  & 
If  SELF-EST="VERY  WELL"  & 
If  SELF-*ii:ST="VERY  WELL"  & 


WRNO=  7  or  8,  then  ACCUR= 
WENO=  5  or  6,  then  ACCUR= 
WRNO=  3  or  4,  then  ACCUR= 
WRNO=  1  or  2,  then  ACCUR= 
WRNO=  0,  then  ACCUR=  4. 


SELF-EST;   SELF-ESTIMATES  OF  ONE'S  OWN  UNDERSTANDING 

WRNO;  THE  NUMBER  OF  WRONG  ANSWERS  IN  A  LESSON 
 CEAI^GE;  0  to  .81 

ACCUR;  ACCURACY  OF  SELF-ESTIHATES  OF  UNDERSTANDING 
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APPB0VRIATEMZS8  OF  THB  U8AQB  OF  OOMTSOL  OPTIOMS 


If 

REVIEWNO 

-  WRNO 

>= 

0, 

then 

APPRO 

4. 

If 

REVIEWNO 

-  WRNO 

-1 

or  -2, 

then 

APPRO  = 

3. 

If 

REVIEWNO 

-  WRNO 

-3 

or  -4, 

then 

APPRO  = 

2. 

If 

REVIEWNO 

-  WRNO 

-5 

or  -6, 

then 

APPRO  = 

1. 

If 

REVIEWNO 

-  WRhO 

-7 

or  -8, 

then 

APPRO  = 

0. 

REVIEWNO;  THE  NUMBER  OF  REVIEW  OPTIONS  USED 

WRNO;   THE     NUMBER  OF  WRONG  ANSWERS  IN  A  LESSON 
 (RANGE;   0  TO  8) 

APPRO;  APPROPRIATENESS  OF  THE  USAGE  OF  CONTROL  OPTIONS 


EFFBCTIVEMESS  OF  THE  USAGB  OF  COHTROL  OPTZOVS 

EFFECTIVENESS  -   (ACCUR  +  APPRO)  /  2 

ACCUR;  accuracy  OF  SELF-ESTIMATES  OF  UNDERSTANDING 
APPRO;  APPROPRIATENESS  OF  THE  USAGE  OF  CONTROL  OPTIONS 
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APPENDIX  3 


QUESTIONS  UND  PROMPTS  ADAPTED  FROM  THE  HETACOHPONENTIAL 
INTERVIEW: 


QUESTION  #1: 

When  Albert  was  6  years  old,  his  sister  was 
3  times  as  old  as  he. 

Now  he  is  10  years  old  and  he  figures  that 
his  sister  is  30  years  old. 

How  old  do  you  think  his  sister  will  be 
when  Albert  is  12  years  old? 

PROMPTS ; 

1.  Do  you  have  to  watch  out  for  mistakes 
when  you  do  this  problem? 

2.  Is  there  something  in  the  problem 

that  could  trick  you  if  you  were  not  careful? 

3.  Is  Albert  right  when  he  figures  that 
when  he  is  10  years  old  his  sister  is 
30  years  old? 

4.  Will  his  sister  always  be  3  times 
as  old  as  Albert? 

Is  that  a  mistake?    Should  you  multiply 
or  add  years? 

5.  Don't  make  a  mistake. 

When  Albert  was  6,  his  sister  was  18. 
12  years  older. 

What  wou7.d  his  sister  be  1  year  later, 
when  Albert  was  7  ? 

(19)     12  years  older.    One  year  later? 
(Continue  if  child  does  not  understand, 
until  solve  10/22  problem  of  Albert* s.) 
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QUESTION  «2: 


Every  classroom  in  Mary's  school  will  have 
2  girls  for  every  boy. 

Mary  found  out  that  there  will  be  8  girls 
in  her  class,  so  she  figured 
there  would  be  16  boys. 

Her  sister's  class  will  have  10  girls. 

How  many  boys  do  you  think  will  be 
in  her  sister's  class? 

PROMPTS ; 

1.  Do  you  have  to  watch  out  for  mistakes 
when  you  do  this  problem? 

2.  Is  there  something  in  the  problem 

that  could  trick  you  if  you  were's  careful? 

3.  Is  Mary  right  when  she  figures  that 
there  will  be  16  boys  in  her  class? 

4.  Will  a  classroom  with  2  girls  for  every  boy 
have  2  t:imes  as  many  boys? 

Is  that  a  mistake?    Or  half  as  many  boys? 

5.  Be  careful  not  to  make  a  mistake. 

If  there  were  2  girls,  and  there  are  2  girls 

for  every  boy,  then  there  would  be  1  boy. 

Half  as  many  boys. 

If  there  were  4  girls,  then  there 

would  be  2  boys. 

Half  as  many  boys. 

(Continue  if  child  does  not  understand 
until  the  8/4  problem  of  Mary's  is  solved.) 
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QUESTION  #3: 

At  our  school  the  track  is  in  a  circle 
with  six  flags  spaced  equally  around  it. 

Yesterday,  Frank  started  at  the  first  flag 
and  raced  around  the  circle. 

He  took  2  minutes  to  get  to  the  third  flag. 

He  thouqht  it  would  take  him  2  more  minutes  to  get 
to  the  sixth  flag  if  he  kept  on  at  the  same  rate. 

How  long  did  it  take  him  to  get  all  of  the  way 
around  the  track? 

PROMPTS  2 

1.  Do  you  have  to  watch  out  for  mistakes 
when  you  do  this  problem? 

2.  Is  there  something  in  the  problem 

that  could  trick  you  if  y^^u  were  not  careful? 

3.  Is  Frank  right  when  he  figures  that 

it  would  take  him  2  more  minutes  to  get 
to  the  sixth  flag? 

4.  Is  it  the  same  distance  from  the  first  to 

the  third  as  from  the  third  to  the  sixth  flag? 
Is  that  a  mistake?    Where  does  he  start? 
How  many  flags  is  it  to  the  third  flag  starting 
at  the  first  flag? 

5.  Don't  make  a  mistake. 

Pretend  Frank  starts  at  the  first  flag, 

then  goes  to  the  second,  then  the  third  flag. 

That  takes  him  2  minutes. 

How  long  will  it  take  him  to  get  to  the  fourth, 
then  the  fifth  flag? 
(The  sixth?) 
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QUESTION  «4: 

Betty  used  to  weigh  32  pounds 
standing  on  one  foot* 

Frank  said  she  would  have  weighed 
64  pounds  standing  on  both  feet* 

Now  Betty  weights  40  pounds  standing  on  one  foot. 

How  much  do  you  think  she  waighs  now 
standing  on  both  feet? 

PROMPTS; 

1.  Do  you  have  to  watch  out  for  mistakes 
when  you  do  this  problem? 

2.  Is  there  something  in  the  problem 

that  could  trick  you  if  you  were  not  careful? 

3.  Is  Frank  right  when  he  figures  that  Betty  would 
weigh  64  pounds  standing  on  both  feet? 

4.  Will  people  always  weigh  twice  as  much 
standing  on  2  feet  as  they  do  standing 
on  1  foot? 

Is  that  a  mistake? 

Does  that  change  what  they  would  weigh? 
5*  Don't  make  a  mistake. 

Would  you  get  heavier  standing  on  1  foot, 

then  2  feet,  then  maybe  on  your  hands  and  feet? 

Does  that  change  how  much  you  weigh? 
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Introduction 


Personalizing  the  context  of  instruction  has  been  found 
to  facilitate  the  reading  comprehension  of  fourth-grade 
children  who  had  been  previously  identified  as  poor  readers 
^ Bracken,  1982) .     There  is  also  evidence  that 
personalization  increases  the  continuing  motivation  of  high 
school  students  to  return  to  task  (Herndon,  1987) . 

Why  should  personalizing  the  context  of  instruction 
affect  learner  outcomes?    Miller  and  Kulhavy  (in  press) 
hypothesized  that  personalization  improves  memory  by 
increasing  the  associative  strength  during  encoding  of  the 
personalized  material  e*..d  closely  related  content  in  a 
prose  sentence.     In  a  second  experiment  extending 
personalization  to  connective  text,  Miller  and  Kulhavy  (in 
press)  permitted  subjects  to  encode  information  in  terms  of 
their  own  unique  experiences,  again  incorporating  the 
example  that  best  fit  the  referent  stimulus.     In  both 
studies,  they  found  that  incorporating  personalized 
representations  during  encoding  led  to  significantly 
greater  recall  of  related  information. 

According  to  the  National  Assessment  of  Educational 
Progress  (1975) ,  solving  story  problems  is  a  major  weakness 
in  the  mathematical  achievement  of  students.     In  analyzing 
the  results  of  the  National  Assessment  of  Educational 
Progress  (NAEP)  ,  Carpenter,  Cobum,  Reys,  and  Wilson  (1975) 
found  that  only  31%  of  a  national  sample  of  13  year  olds 
could  correctly  solve  a  one-step  word  problem  involving  a 
1-digit  divisor  and  a  4--digit  dividend.    As  a  reaction  to 
the  findings  of  the  NAEP,  the  National  Council  of 
Supervisors  of  Mathematics  (1977)  placed  solving  word 
problems  first  on  its  list  of  the  ten  most  basic  skills  in 
mathematics. 

Since  learning  to  solve  word  problems  is  well 
documented  as  being  difficult  (Ballew  &  Cunningham,  1982; 
Carpenter  et  al.,  1975;  Marshall  &  Smith,  1987;  Wright  & 
Wright,  1986),  a  number  of  researchers  have  attempted  to 
explain  the  causes  of  such  difficulties.     De  Corte, 
Verschaffel,  &  De  Win  (1985)  found  that  ambiguously  stated 
problems,  and  not  a  lack  of  computational  skills  per  se, 
significantly  contribute  to  the  inability  of  children  to 
comprehend  the  tei^t  of  word  problems. 

Both  reading  ability  (Ballew  &  Cunningham,  1982)  and 
computational  ability  (Muth,  1984)  have  been  found  to  be 
significant  factors  in  contributing  to  skill  in  solving 
word  problems. 

Personalizing  of  word  problems  appears  to  ameliorate 
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some  of  the  difficulties  youngsters  have  in  solving  such 
problems.     Kintsch  (1986)  and  Hudson  (1983)  report  that 
word  problems  containing  abstract  situations  or  actions  are 
twice  as  difficult  (39%  as  opposed  to  79%)  for  children  to 
solve  as  are  word  problems  incorporating  actions  or 
situations  that  are  concrete  and  familiar  to  the  child. 
Wright  and  Wright  (1986)  administered  personalized  test 
items  to  fourth  graders  and  found  significant  improvement 
in  students'  ability  to  select  correct  arithmetic 
operations,  but  little  effect  on  their  computation  skills. 
Ross  and  Anand  (1987)  and  Anand  and  Ross  (1987)  found  that 
a  personalized  context  of  instruction  embedding  familiar 
items,  such  as  learner  interests  and  background,  increased 
the  achievement  of  fifth  and  sixth  graders  to  solve 
verbally  stated  math  problems.     However,  subjects  in  both 
studies  by  Ross  and  Anand  were  enrolled  in  a  university 
affiliated  school.    Thus,  it  may  be  inappropriate  to 
generalize  their  findings  to  children  attending  schools  in 
lower  socioeconomic  areas. 

Personalizing  the  context  of  instruction  has  been 
explored  both  with  groups  and  with  individuals.     In  a 
program  on  conditional  syllogisms,  Herndon  (1987) 
personalized  the  context  of  practice  items  based  on  the 
most  common  interests  of  the  subject  pool,  rather  than  on 
individual  interests.    Herndon  (1987)  found  that 
personalization  significantly  affected  the  continuing 
motivation  of  high  school  seniors  to  return  to  task,  but 
did  not  significantly  improve  their  achievement.  Lopez 
(1989)  personalized  word  problems,  using  individual 
responses  to  a  biographical  inventory.    Her  results  yielded 
significant  differences  favoring  the  personalized  treatment 
in  both  achievement  and  student  attitude,  but  no 
significant  differences  in  continuing  motivation. 

The  rationale  for  using  group  instead  of  individually 
personalized  context  is  one  of  expediency  and  efficiency. 
Incorporating  group  interests  involves  the  development  of  a 
lesson  or  lessons  that  can  be  used  for  the  entire  class- 
Individual  interests  require  that  separate  lessons  be 
generated  for  each  student.    The  group  personalization 
should  take  considerable  less  time  to  generate  than  the 
individual  one.    Consequently,  it  is  important  to  determine 
if  the  group  method  is  as  effective  as  the  individualized 
one. 

Research  on  sex  differences  in  mathematics  achievement 
has  produced  mixed  results  (Marshall,  1984).    Maccoby  and 
Jacklin  (1974)   found  no  sex  differences  until  age  14,  but 
differences  favoring    males  after  that  age*    Results  from 
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the  1979  NAEP  data  showed  no  significant  sex-related 
differences  among  9,  13,  and  17  year  olds  in  solving  word 
problems  (Linn  &  Petersen,  1986). 

In  a  review  of  the  1978  NAEP  results,  Valverde  (1984) 
found  that  Hispanic  children  were  9  percentage  points  below 
the  national  average  (55%  to  46%)  in  mathematics 
achievement  at  age  9.     By  age  17,  Kispanics  had  dropped  to 
12  percentage  points  below  the  national  average  (60%  to 
48%) .    Matthews,  Carpenter,  Lindquist,  Silver  (1984) 
reported  similar  results  from  the  1982  NAEP  data. 

Despite  these  findings,  few  studies  have  examined  ways 
to  improve  the  mathematical  performance  of  Hispanic 
students.     In  one  such  study,  Hannafin  (1983)  found  a  trend 
toward  superior  performance  for  Hispanic  sixth  graders 
receiving  an  instructional  system  designed  to  provide 
hierarchically  sequenced  sets  of  computational  skills  over 
Anglos  who  had  received  traditional  instruction.  This 
trend  occurred  only  on  instructional  tasks  which  were 
relatively  novel  or  independent  of  prerequisite  skills. 
Lopez  (1989)  investigated  the  effect  of  individualized 
personalization  on  the  performance  of  Hispanic  eighth 
graders  on  mathematics  word  problems.    Her  data  revealed 
significant  difference!^  favoring  personalized  over 
non-personalized  instruction  on  both  student  achievement 
and  attitudes. 

The  primary  purpose  of  this  study  was  to  investigate 
the  effects  of  three  levels  of  personalization  (individual, 
group,  non-personalized)  on  the  achievement  of  seventh 
grade  Hispanic  boys  and  girls  on  mathematics  word 
problems.    Dat'i  were  also  collected  on  subjects'  attitudes 
toward  the  instruction.    A  pretest  was  employed  to  control 
experimentally  for  potential,  initial  achievement 
differences  by  treatment  or  sex.     Subjects  were  blocked  by 
sex  on  the  basis  of  their  pretest  scores,  then  randomly 
assigned  within  each  block  to  the  three  treatments. 

Method 

Subi  ects 

One  hundred  twenty-three  seventh  grade  Hispanic 
students  from  a  rural  junior  high  school  with  a  primarily 
Hispanic  enrollment  near  Phoenix,  Arizona  were  the  final 
group  of  subjects  for  this  study.    Subjects  were  from  10 
math  classes  taught  by  four  teachers. 
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Materials 

Biographical  inventory >    A  biographical  inventory  based 
on  the  work  of  Anand  and  Ross  (1987)  and  Ross  and  Anand 
(1987)  was  designed  for  this  experiment.    Respondents  were 
asked  questions  regarding  their  background  and  interests. 
Topics  included  the  names  of  friends  and  favorite  places, 
foods,  objects,  activities,  and  events. 

Instructional  lesson.      Three  parallel  versions  of  a 
unit  on  division  of  whole  numbers  were  developed.  The 
non-personalized  version^  serving  as  a  template,  was 
written  first.    The  individual  and  group  versions  were  then 
written  with  nouns^  pronouns,  prepositional  phrases,  and 
dates  inserted  as  appropriate  in  the  original 
non-personalized  context.     Feuniliar  items,  including  birth 
dates  and  city  of  residence,  were  derived  from  Biographical 
Inventory  items. 

Variations  in  treatments  were  manipulated  by  changing 
the  referents  of  example  and  practice  problems.  However, 
the  numerical  values  and  methods  of  measurement  (e.g., 
number  or  weight)  were  identical  for  all  treatments. 

The  non-personalized  treatment  used  tne  standard 
referents  found  in  math  textbooks.    An  example  of  a  typical 
non-personalized  referent  follows: 

Jim  expects  36  friends  to  come  to  a  party. 
If  each  friend  will  get  a  6-ounce  (oz) 
serving  of  soda,  how  many  12-oz  cans  of  soda 
are  needed? 

In  the  individual  treatment,  nouns  and  pronouns 
were  replaced  or  modified  with  the  familiar  items 
students  provided  in  the  Biographical  Inventory: 

Gabriel  expects  36  friends  to  come  to  his 
birthday  party  on  March  7.     If  each  friend 
will  get  a  6-ounce  (oz)  serving  of  Dr >  Pepper . 
how  many  12-oz  cans  of  Dr>  Pepper  are  needed? 

The  instructional  lessons  for  the  individual 
condition  were  computer  generated  so  as  to  provide  a 
feasible  way  of  personalizing  the  lesson  context  for 
individual  experiences  and  interests. 

The  group  treatment  was  based  on  the  responses  of 
the  majority.    Nouns,  pronouns,  or  noun  modifiers 
were  replaced  with  the  most  familiar  objects,  places, 
and  persons  the  majority  of  students  provided  in  the 
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Biographical  Inventory.    An  example  of  how  variations 
in  the  group  treatment  were  manipulated  follows: 

Miguel  expects  36  friends  to  come  to  his  birthday 
party  in  December.    I:c  each  friend  will  get  a 
6-ounce  (02)  serving  of  Pepsi ,  how  many  12-02  cans 
of  Pepsi  are  needed? 

The  lesson  covered  procedures  in  solving  one-step  and 
two-step  word  problems  involving  division  of  whole 
numbers.     Instruction  on  the  strategy  for  solving  both  item 
types  (one-step  and  two-step  problems)  contained  the  rule 
and  its  application  with  appropriate  examples.  Practice 
problems,  three  for  each  item  type,  followed.    Each  of  the 
eight  problems  was  placed  on  a  separate  page  with  enough 
open  space  to  allow  students  to  work  the  problem  directly 
or  the  page.     Answers  to  all  practice  problems  were 
provided  at  the  end  of  the  lesson  booklet. 

Review.      Prior  to  administration  of  the  posttest,  rhe 
seven-page  review  over  the  lesson  content  was 
administered.     Three  versions  of  the  review  were  developed 
(individual,  group,  and  non-personalized)  to  parallel  the 
lesson  content.    The  review  contained  a  summary  of  rules 
and  procedures  for  each  item  type    and  four  practice 
problems,  two  for  each  item  type.    As  in  the  instructional 
material,  answers  to  all  problems  appeared  at  the  end  of 
the  booklet. 

Procedure 

The  Biographical  Inventory  was  administered  to  students 
three  weeks  prior  to  administration  of  treatments.  Besides 
biographical  data,  information  regarding  favorite  objects, 
places,  persons,  and  events  were  obtained. 

Subjects  were  blocked  by  sex  then  randomly  assigned 
within  blocks  to  one  of  three  levels  of  personalization 
(individual,  group,  or  non-personalized) . 

The  three  versions  of  the  instructional  lessen  were 
administered  on  a  within-class  basis  in  regularly  scheduled 
math  classes  three  weeks  after  the  administration  of  the 
Biographical  Inventory.    The  cover  sheet  on  each  booklet 
contained  the  student's  name,  class  period^  and  task 
related  instructions  which  the  experimenter  read  to  all 
subjects.     Students  were  told  that  they  were  helping  to 
evaluate  a  new  lesson.    Additionally,  they  were  told  that 
after  completing  the  lesson  and  review,  they  would  take  a 
test  and  that  their  test  scores  would  count  toward  their 
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grade . 

The  review  was  given  on  the  following  day.  The 
posttest  was  distributed  to  each  student  imediately  after 
completion  of  the  review  material. 

Criterion  Measures 

Achievement.      The  16-item  constructed-response 
posttest  served  as  the  criterion  measure  for  achievement. 
The  test  coverei  the  seme  mathematics  operations  as  the 
instructional  materials.    It  consisted  o2  eight  one-step 
and  eight  two-step  word  problems.    None  of  the  16  probleius 
involved  remainders.    All  problems  were  in  the 
non-personalized  form.    The  inter-item  reliability  for  the 
posttest,  calculated  in  an  earlier  study  by  Ldpez  (1989) , 
is  .77. 

Attitudes  and  continuing  mc.ivation.      This  4-poini: 
scale,  containing  11  items,  assessed  level  of  interest; 
level  of  difficulty;  number  of  familiar  persons,  places,  or 
things;  perception  of  the  importance  of  the  lesson;  level 
of  importance  attached  to  seeing  one's  name  in  print;  how 
much  subjects  liked  the  lesson;  preference  for  personalized 
and  non-personalized  problems;  and  preference  for  word  and 
nximber  problems.    Open-ended  questions  assessed  what 
subjects  liked  most  and  least  about  the  lesson. 

Design  and  Data  Analysis 

The  experimental  design  was  a  3  I,evel  of 
Personalization  (Individual/  Group/  Non-Personalized)  x  2 
Sex  (Male/  Female)  x  2  Item  Type  (One-Step/  Two-Step) 
factorial  design,  with  item  type  as  a  wi thin-subjects 
variable.    Achievement  data  were  analyzed  by  multivariate 
analysis  of  variance  (MANOVA)  repeated  measures.  Attitude 
data  were  analyzed  by  separate  analyses  of  variance 
(ANOVA) . 

Results 

Achievement 

Table  1  shows  the  mean  scores  by  level  of 
personalization,  sex,  and  item  type.    The  mean  overall 
posttest  scores  by  level  of  personalization  were  9.49  for 
the  individualized  treatment,  9.61  for  group,  and  7.7  3  for 
non-personalized.    The  multivariate  analysis  yielded  a 
significant  difference  for  treatments,  £(2,  117)  =  3.61,  p 
<  .03.    Univariate  analyses  revealed  that  the  significant 
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treatment  effect  occurred  for  two-step  problems,  Z(2,  117) 
=  4.01,  E  <  .021,  but  not  for  one-step  problems. 

Post  hoc  contrasts  using  Newman  Keuls  were  performed  to 
determine  the  source  of  differences  between  treatments. 
Both  the  individualized  mean  and  the  group  mean  were 
significantly  higher  at  the  .05  level  than  the  mean  for  the 
non-personalized  treatment.    These  differences  are 
reflected  in  the  two-step  problem  means  of  3.23  for 
individualized  subjects,  3.44  for  group  subjects,  and  only 
2.06  for  non-personalized  subjects.    The  individualized  and 
group  means  did  not  differ  significantly  from  each  other. 

Overall  mean  scores  by  type  of  test  items  were  6.02  for 
one-step  items  and  2.89  for  two-step  items,  a  highly 
significant  difference,  £(1,  117)  =  288.43,  q  <  .001.  The 
mean  posttest  score  for  females  was  9.06  and  the  mean  for 
males  was  8.71,  a  non-significant  difference. 

Attitudes 

The  mean  attitude  scores,  based  on  a  score  of  1  for  the 
most  positive  response  to  each  item  and  4  for  the  most 
negative,  are  shown  in  TedJle  2.    The  table  reveals  that  the 
overall  mean  scores  across  all  seven  items  were  1.69  for 
the  individualized  treatment^  1.82  for  the  group  treatment, 
and  1.87  for  non-personalized  instruction.    Analysis  of 
variance  of  these  means  yielded  a  highly  significant: 
difference,  £(2,  117)  «  3.12,  p  <  .001.     Post  hoc  contrasts 
using  Newman  Keuls  revealed  that  the  overall  mean  for  the 
individualized  treatment  was  significantly  higher  than 
those  for  both  the  group  and  the  non-personalized 
treatments.    The  group  and  non-personalized  means  did  not 
differ  significantly  from  one  another. 

When  the  seven  questionnaire  items  shown  in  Table  2  are 
considered  individually,  it  can  be  seen  that  significant 
differences  occurred  for  items  2,  3,  6,  and  9.    Post  hoc 
contrasts  showed  that  the  individualized  treatment  mean  was 
significantly  different  from  the  non-personalized  mean  (p  < 
.05)  on  each  of  these  items.     Individualized  subjects 
reported  that  the  lesson  was  more  difficult,  there  was  more 
familiar  content,  they  liked  the  lesson  better,  and  they 
preferred  personalized  content.    Means  between  the 
individual  and  group  treatments  and  the  group  and 
non-personalized  treatments  did  not  differ  significantly 
from  one  another  on  any  of  the  four  items. 

Two  additional  items  on  the  attitude  questionnaire 
assessed  preference  for  math  word  problems  that  were  not 
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personalized  and  preference  to  study  more  math  word  as 
opposed  to  number  problems*    ANOVA  followed  by  Newman  Keuls 
revealed  that  subjects  in  the  individualized  treatment 
(2.72)  had  a  significantly  lower  preference  for 
non-personalized  problems  than  subjects  in  both  the  group 
(2.42)  and  the  non-personalized  (2o39)  treatments,  E(2, 
117)  =  5.05,  E  <  •008*     The  three  treatment  groups  did  not 
differ  significantly  in  their  preference  to  study  more  word 
problems. 

Frequency  of  constructed  responses  to  the  open-ended 
questions  about  what  subjects  liked  most  and  what  they 
liked  least  were  also  tabulated.    Student  responses 
indicated  that  what  they  liked  most  was  the  personalization 
of  the  lessons*     This  factor  was  cited  24  times,  19  by 
subjects  in  the  individualized  treatment  and  5  by  those  in 
the  group  treatment.    Subjects  cited  specific  types  of 
computations,  notably  division  and  multiplication,  as  what 
they  liked  least.    A  total  of  25  responses  fell  into  this 
category,  13  from  the  non-personalized  treatment. 

Discussion 

The  present  research  revealed  a  significant  affect  for 
personalization  on  student  achievement  on  two-step 
problems,  the  more  difficult  problems  in  the  instructional 
unit.    It  seems  likely  that  at  least  two  factors  may  have 
contributed  to  this  effect.    One  is  that  personalized 
subjects  liked  the  lesson  more^  as  indicated  by  responses 
of  individualized  subjects  to  question  6  on  the  attitude 
survey  —  "How  much  did  you  like  the  lesson?"    This  could 
have  caused  them  to  expend  greater  effort  on  the  lesson, 
which  could  explain  the  fact  that  the  individualized 
subjects  reported  the  instruction  to  be  significantly 
harder  (question  2  on  the  attitude  survey) .     If  subjects 
receiving  personalized  instruction  likad  the  instruction 
more  and  invested  more  effort  in  it,  they  could  reasonably 
be  expected  to  demonstrate  improved  performance. 

A  second  possible  contributing  factor  is  that 
personalizing  the  problems  may  have  increased  subjects' 
level  of  understanding  of  them.    When  the  problems  are 
personalized  by  using  familiar  names,  objects,  and 
situations,  the  problem  situation  may  be  more  familiar  to 
the  subjects.    They  may  therefore  be  more  able  to  deduce 
the  necessary  mathematical  operations. 

An  explanation  from  cognitive  theory  suggests  how 
personalization  may  influence  learning  by  affecting 
encoding  and  retrieval.    Miller  and  Kulhavy  (in  press) 
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demonstrated  that  individualized  personalization  increases 
the  retrievability  of  associate  information  by  stimulating 
stronger  and  more  memorable  encoding •     If  personalization 
does  provide  the  learner  with  more  memorable  illustrations 
of  rule  application,  then  more  effective  semantic 
processing  of  the  problem  situation  may  occur. 

Lopez  (1989)  found  in  an  earlier  experiment  that 
individualized  personalisation  of  instruction  was  effective 
in  improving  the  achievement  of  Hispanic  children.  The 
present  results  indicate  that  personalization  is  also 
effective  with  Hispanic  students  on  a  group  basis  as  well 
as  on  an  individual  one.    Thus,  this  finding  extends  to 
group  personalization  the  earlier  results  obtained  with 
individualized  personalization  by  L6pez  (1989)  and  by  Anand 
and  Ross  (1987)  and  Ross  and  Anand  (1987) . 

Males  and  females  in  the  present  study  did  not  differ 
significantly  in  their  overall  performance  or  their 
performance  on  particular  types  of  problems.    This  finding 
is  consistent  with  the  results  of  the  National  Assessment 
of  Educational  Progress  (NAEP)  from  1973,  1978,  and  1982. 
The  NAEP,  a  comprehensive  national  test  administered  to  a 
very  large  sample,  mainly  uses  constnicted-response  items 
for  word  o^oblems  (NAEP,  1977) ,  as  did  the  present  study. 
The  NAEP  cata  consistently  yielded  no  significant 
sex--relatad  differences  among  students  at  ages  9  and  13  in 
solving  math  word  problems  (Linn  &  Petersen,  1986;  NAEP, 
1983).     Several  studies  published  in  the  1970s  also 
reported  no  sex-related  differences  in  mathematics 
achievement  in  the  elementary  grades  (Hilton  &  Berglund, 
1974;  Maccoby  &  Jacklin,  1974). 

Other  researchers,  however,  report  sex*- related 
differences,  generally  favoring  males  in  mathematics 
achievement*    Fennema  and  Sheznnan  (1977),    Hilton  and 
Berglund  (1974),  and  Maccoby  and  Jacklin  (1974)  all  report 
significant  differences  favoring  males  starting  at  grade 
7.    Benbow  and  Stanley  (1982)  report  significant 
differences  favoring  males  with  mathematically  gifted  and 
highly  motivated  seventh  and    eighth  graders.    All  of  these 
results  favoring  males  were  from  research  using 
multiple-choice  tests. 

The  variations  in  findings  regarding  sex-related 
differences  may  be  a  function  of  either  the  differing 
nature  of  the  tests  (constructed-response  versus  multiple 
choice)  or  of  the  differing  subject  populations.    The  most 
comprehensive  sample  across  time,  grade  levels,  number  of 
subjects,  and  geographic  sites  was  from  the  NAEP  studies, 
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which  consistently  yielded  no  sex-related  differences  on 
primarily  constructed- response  items.     That  boys  performed 
better  than  girls  on  several  studies  involving 
multiple-choice  tests  may  relate  to  the  fact  that  these 
tests  require  somewhat  different  cognitive  operations  than 
constructed-response  tests.     The  correct  answer  can  often 
be  derived  in  the  former  by  examining  the  response  choices 
and  estimating  the  most  reasonable  choice,  whereas 
constructed-response  tests  require  computation  of  the 
correct  answer.     It  is  possible  that  males  are  better  at 
the  estimating  skill,  but  not  on  the  computations  required 
in  constructed-response  tests. 

The  attitude  data  generally  favor  individualized 
personalization.     Subjects  under  individualized 
personalization  had  significantly  more  positive  overall 
attitudes  than  those  under  the  other  two  treatments. 
Individual  sixbjects  liked  the  lesson  better,  recognized 
more  familiar  content,  preferred  personalized  problems,  and 
had  a  lower  preference  for  the  prospect  of  non-personalized 
problems  in  future  math  content.     The  most  plausible 
explanation  for  the  unexpected  finding  that  subjects  in  the 
individualized  treatment  reported  the  lesson  to  be  more 
difficult  would  seem  to  be  that  the  personalized  subjects 
expended  greater  effort  on  the  lesson  because  of  the 
personalization. 

An  interesting  aspect  of  the  high  attitude  ratings  for 
individualized  personalization  is  that  they  were  obtained 
with  content  that  is  generally  unpopular  with  students. 
Story  problems,  especially  multi-step  problems,  are 
considered  by  students  to  be  difficult  (Ballew  £ 
Cunningham,  1982;  Marshall,  1984;  Marshall  &  Smith,  1987) 
and  boring  (Cheek  &  Castle,  1981) .     Nevertheless,  the 
attitudinal  responses  of  personalized  subjects  in  this 
study  indicated  that  the  instruction  was  interesting  and 
that  they  liked  it. 

The  present  results  have  implications  for  educational 
practice  generally.    The  simplest  application  is  for 
instructors  to  learn  the  interests  of  their  students, 
either  through  a  formal  survey  of  interest  or  more  informal 
means,  and  incorporate  them  into  instruction  whenever 
possible.     The  data  from  this  study  suggest  that  use  of 
common  interests  of  the  group  in  this  manner  is  also 
effective.     More  sophisticated  application  could  involve 
the  design  of  computer  software  that  incorporates 
individual  or  group  interests  into  word  problems  and  other 
practice  material  while  maintaining  the  integrity  of  the 
content  ty  be  learned. 
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Both  the  present  study  and  the  earlier  research  by 
Lopez  (198S)  indicate  that  personalization  in  mathematics 
instruction  is  effective  in  improving  the  performance  of 
Hispanic  children,  especially  with  two-step  math  problems. 
Both  studies  were  conducted  over  relatively  brief  time 
periods  of  two  instructional  lessons  each.  Further 
research  should  be  conducted  to  investigate  the  extent  to 
which  positive  effects  would  be  sustained  over  an  extended 
time  period  and  with  different  subject  matter.  Research 
investigating  these  issues  should  help  to  clarify  the 
conditions  under  which  personalization  of  instruction  is 
most  effective  in  maximizing  improved  classroom 
performance . 
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Table  1 

Means  and  Standard  Deviations  of  Posttest  Scores  bv  Level  of 
Personalization.  Sex,  and  Test  Item  Type 


Level  of  Personalization 


Individualized  Group  Non-Personalized 


Sex  1-Step  2 -Step  1-Step  2 -Step  1-Step  2 -Step  Total 
Females 

Jl  6.23  3.36  6.24  3.38  5.62  2.46  9.06 

SD  1.85  2.48  1.70  2.65  2.00  2.23  3.80 

Males 

M  6.29  3.09  6.07  3.53  5.70  1.65  ^.71 

SD  1.49  2.36  1.58  2.10  1.42  2.25  3.40 

Total 

M  6.26  3.23  6.17  3.44  5.67  2.06  8.90 

SD  1.66  2.40  1.63  2.41  1.71  2.24  3.63 
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Table  2 

Questionnaire  Responses  by  Treatment 


Quest  i  onna  i  re  Non- 

Item  Individ    Group    Pers    £~value  E""lGvel 


1*  Interesting- 

Borina  1.93        1.89      2.07         .85  NSD 


2.  Easy-Hard  2.02        1.75      1.75      3.30  <.04 


3.  Familiar 

Content  1.53         1.81      2.16       5.80  <.004 


4 .  Importance 
of  Problem 

Solving  1.28        1.44       1.23       1.69  NSD 


5 .  Importance 
of  Name  in 

Print  2.07        2.25      1.93         .97  NSD 


6.  Liking  for 

Lesson  1. 63         1.94       2.02       3 .81  <. 025 


9.  Preference  for 
Personalized 

Problems  1.37        1.69       1.93       5.67  <.004 


GRAND  MEAN  1.69         1.82       1.87       3.12  <.001 


Note.     Questions  7  and  8  were  open-ended  items. 


413 

ERIC  414 


Title: 

Learner  Preferences  for  Varying  Screen  Densities 
Using  Realistic  Stimulus  Materials  with  Single  and 

Multiple  Screen  Designs 


Authors: 

Gary  R.  Morrison 
Steven  M.  Ross 
Charles  W.  Schultz 
Jacqueline  K.  O'Dell 


415 

415 


Screen  Preferences 
2 


Abstract 


Learner  preferences  for  varying  screen  density  levels 
were  examined  using  multiple  screen  designs  (high  external 
validity)  and  single  screen  designs  (high  internal 
validity).     When  viewing  multiple  screens  for  each  design  in 
Study  I,   subjects  indicated  the  highest  preference  for 
medium  density  screens  while  tending  to  select 
higher-density  over   lower -dens  ity  screens   in  individual 
comparisons.     When  viewing  only  the  first  8cre<^n  of  each 
density  level  in  Study  II,   subjects  again  expressed 
preferences  for  higher-density  over  lower -density  designs . 
Suggestions  are  provided  concerning  the  use  of  realistic  and 
nonr ealis t ic  content  for  the  s timulus  materials  as  well  as 
Implications   of  using  externally  and  internally  valid  screen 
designs  for  future  resear  h  on  computer-based  instruction 
screen  design 
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The  continuing  expansion  of  the  microcomputer  into 
schools .   businesses ,   hospitals .   and  homes  has  created  a 
market  for  instructional  software  ranging  from  beginning 
mathematics   programs   to  sophisticated  simulations  of 
hospital  emergency  room  events .     A  review  of  these 
instructional  packages  indicates  the  integration  of 
graphics,   sound,   animation,   and  both  effective  and  poor 
applications  of  instructional  design  (Bork  1987:  Burke. 
1981:   Keller.   1987).     Unfortunately,   software  designers  have 
a  tendency  to  design  computer  screens  based  on  principles 
derived  from  print  based  research  (e.g..   Hartley,   1985),  yet 
a  comparison  of  the  attributes  of  the  two  media  reveals 
several  important  differences.     Computer  displays  (a)  are 
limited  to  one  page  at  a  time,    (b)  have  restricted  backward 
paging  and  review,   (c)  are  limited  to  layouts   of  40  or  80 
columns   by  24  rows,   (d)  provide  limited  cues  as  to  lesson 
length.   Ce)  are  typically  limited  to  one  typeface  and  one  or 
two  typesizes.   and  (f)  offer  relatively  poor  resolution.  In 
contrast  to  the  printed  page,   the  computer  has  the 
capability  to  generate  dynamic   "pages"   (e.g..  windows, 
screen  building,   and  animation),  which  can  be  increased  in 
number  with  a  relatively  smaller  effect  on  distribution 
c  OS  t  s  . 

Computer  Screen  Design 

The  literature  on  computer  screen  design  tends  to 
follow  one  of  two  approaches.     The  first  approach  focuses  on 
typographical  variables   that  the  designer  can  manipulate  to 
create  an  effective  screen  design.     Based  on  research  and 
subj  ective  views ,   several  authors  have  recommended  that 
displays  feature  liberal  white  space,   double  spacing,  a 
standard  ASCII  typeface,   and  left-justified  text   (Allessi  & 
Trollip,   1985:  Bork.    1984.   1987:  Grablnger ,   1983:   Heines . 
1984;-  Hooper  &  Hannafin.   1986).     Given  the  recent 
introduction  of  bit  mapped  graphics,   information  concerning 
the  manipulation  of  typefaces,   type  size,   leading,  and 
a irailar  typographical  variables  will  also  bee ome  more 
acces  sible . 

A  second  approach  to  computer  screen  design  is  the 
manipulation  of  the  content.     One  such  method  is  chunking 
the  material  into  meaningful  thought  units  which  are  then 
presented  with  blank  (white)  spaces  bordering  each  (Bassett. 
1985:   Feibel.   1984;  Grabinger     1983).     Although  Falio  and 
DeBloois  (1988)  suggest  chunking  as  an  effective  means  of 
designing  displays,   research  on  chunking  and  similar  methods 
have  failed  to  show  clear  advantages  under  either  print  or 
CBI   (cf.   Basset.    1985;  Carver.    1970;   Fiebel.   1984:  Gerrel  & 
Mason.    1983:  0' Shea  &  Sinclair.   1983).     It  seems  important 
to  consider  that  chunking  does  not  change  the  instructional 
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content;  rather,   it  changes  the  way  the  content  is  presented 
on  the  screen.     In  contrast,  Morrison.  Ross.  &  0' Dell  (1988; 
Ross,  Morrison.  &  0* Dell ,   1988)  varied  the  text  density  in 
presentations  as  represented  by  the  length  of  the  materials, 
redundancy  of  explanations,   and  depth  of  contextual  support 
for  main  ideas       They  found  that  the  lower  density  text  w^\s 
read  significantly  faster  than  conventional  text  with  no 
reduction  in  achievement.     Subjects  also  chose  lower-density 
over  higiier-density  text  65^  of  the  time. 

In  suraniary,   these  two  approaches,  typographical 
variables  and  content  manipulation,   have  provided  useful 
guidelines  for  screen  design;   however ,   they  have  not 
addressed  the  issue  of  how  much  information,  "screen 
density,"   the  expository  frame  should  contain.     For  example, 
the  International  Reading  Association  Computer  and 
Technology  Reading  Committee  (1984)  recommends  using  "clear 
and  legible"   displays  with  "appropriate  margins  and 
interline  spacing" ,   but  provides  no  operational  guidelines 
or  specifications  to  define  these  qualities.     To  provide 
designers  with  clearer  recommendations  for  optimum  density 
levels ,   the  screen  density  construct  must  be  oper ationalizea 
and  precisely  defined. 
Screen  Density  as  a  Design  Variable 

One  method  of  evaluating  screen  designs   is  to  calculate 
the  density  of  the  total  screen  by  determining  how  many  of 
the  screen  spaces  contain  a  character  or  are  adjacent  to  a 
character  (Tullis ,   1983).     It  is  assumed  that  instructional 
displays  are  relatively  uniform  in  density  due  to  the  use  of 
prose ,   as  compared  to  instrumentation  readout  displays  which 
often  chunk  the  information  into  different  sections  of  the 
screen . 

Human  factors  research  suggests  that  performance  error 
rates  increase  as  the  density  of  a  display  increases  (Burns . 
1979,  Coffey,   1961:  Mackworth.   1976;  Ringel  and  Hammer, 
1964).     Research,   however,   on  the  upper  limit  of  screen 
density  has  yielded  disparate  recommendations  ranging  from 
ISH  (Danchak.    1976)  to  31.2%  (Smith.    1980,   1981,    1982)  all 
the  way  to  60%  (NASA,   1980).     Two  reasons  for  these 
inconsistencies  are  suggested.     First,   the  displays  have 
often  involved  instrumentation  screens  and  information 
displays  that  are  too  esoteric  (i.e..  unique  to  a  specific 
environment)  to  be  generalizeable  to  instructional  screens. 
Second,   several  of  the  studies  have  used  isolated  screen 
displays  of  unrealistic  stimulus  materials  which  have  low 
ecological  validity  (see  Ross  fit  Morrison,  1989). 

In  a  realistic  lesson  the  number  of  frames  increases  as 
the  amount  of  white  space  increases   (i.e..   screen  density  xs 
decreased).     Thus,  manipulation  of  screen  density  in  a 
single  frame  presentation  fails  to  account  for  the 
concomitant  effect  of  the  increase  or  decrease  in  the  number 
of  screens  required  to  read  the  same  information. 
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According] y .  in  contract  with  previous  research,   the  present 
study  was  designed  to  examine  learner  preferences  for 
different  screen  densities  used  to  present  a  fixed  amount  of 
material.     Depending  on  the  particular  density  level 
represented,  froni  one  to  four  frames  of  Information  were 
required  to  view  the  content. 

Another  concern  In  Investigating  screen  density 
preferences  Is   the  possible  Influence  of  the  type  of 
material  presented  on  how  different  screen  designs  are 
viewed.     For  example ,   Grablnger ' s   (1983)  evidence  for 
supporting  low  density  screens   was   obtained  using  a 
typographical  notation  developed  by  Tw^man  (1981)  to  crcsto 
a  content-free  screen  r epr ecentatl on  of  a  CBI  screen. 
Specifically,   screens  were  designed  with  x' s  and  o' s  to 
resemble  actual  lessons.     However,   when  reacting  to  displays 
of  abstract  or  artificial  materials,  subjects  may  prefer 
wide  margins  and  other  lower  density  attributes  due  to  the 
greater   sallency  of  aesthetic  properties  when  there  la  no 
need  to  understand  the  content.     In  contrast,   judgments  of 
realistic  materials  would  appear  to  demand  greater  awareness 
of  and  reliance  on  contextual  properties   (e.g..  proximal 
supporting  text)  that  helps  to  Increase  the  meaning  of  the 
Information  being  read.     Thus .    It   Is  not  clear  that 
)reference8  for  low-density  screens  similarly  apply  to 
realistic  lesson  materials,   especially  since  the  low-density 
designs   present  the  material  In  smaller  thought  units  and 
consequently  also  necessitate  an  Increased  number  of  lesson 
frames . 

Accordingly,    to  extend  Grablnger* s  (1983)  research,  the 
present  study  used  realistic  materials  from  an  actual  course 
In  the  subjects*    academic  program.     Wo  expected  that  with 
fixed  content  and  realistic  displays,   preferences  for 
lower -dens  1 ty  screens  would  not  be  as   high  as  previous 
research  in  the  instructional  design  literature  generally 
suggests.     A  third  research  Interest  was   the  preferences  of 
users  differing  In  degree  of  CBI  experience,   namely  graduate 
instructional  design  students  versus  undergraduate  education 
students . 
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Study  I 


Method 


Subjects  and  Design 

Subj  ect.    were  23  graduate  and  23  undergraduate 
education  majors  (29  females  and  17  males)  who  volunteered 
to  participate  In  the  study.     A  paired-comparison  design 
(Nunnally,   1967)  was  employed  involving  a  total  of  six 
unique  pairings  of  four  density  levels  presented  on  an  Apple 
lie  monochrome  screen.     For  each  of  the  six  comparisons, 
subjects  were  presented  with  two  different  screen  designs 
and  asked  to  indicate  their  preference.     The  six  comparisons 
and  the  two  density  levels  within  each  were  presented  in  a 
random  order.     To  begin  the  session,   subjects  completed  a 
9-item  attitude  survey  presented  on  the  computer.     They  were 
then  presented  the  six  comparisons  and  asked  to  indicate 
their  preference  on  each. 

Materials 

Materials  used  in  this  study  are  described  below  in  the 
order  in  which  they  were  used. 

Profile  Data.     A  9-iteTO  survey  was  used  to  determine 
subjects*   attitudes  towards  using  the  microcomputer.  Each 
Item  was  presented  on  the  computer  screen.     Subjects  reacted 
to  each  using  a  five-point  scale  with  5  representing  the 
most  positive  reaction.     Six  of  the  items  concerned  the 
subjects'   attitudes  towards  using  the  microcomputer  lor  work 
or  school.     The  remaining  three  items  concerned  their 
attitudes  towards  learning  how  to  use  a  microcomputer. 

Screen  Displays.     A  single  screen  selected  from  a 
computer-based  lesson  used  in  previous  studies  on  text 
density  (Morrison.   Ross.   &  ODell.   1938:   Ross  et  al . .  i088) 
was  selected  as  the  basic  content  for  this  study  (see  Figure 
1).     The  material  was  from  an  instructional  unit  on 
statistics   (Ross.   1983)  currently  used  in  an  undergraduate 
education  course  at  the  same  university  in  which  the  study 
was  conducted.     To  determine  the  screen  density  of  the  core 
frame,   all  characters  and  spaces  contiguous  to  the 
characters  were  counted  and  then  divided  by  the  total  number 
of  characters  the  scroen  could  display  (960  for  a  40  column 
X  24  row  format).     The  resultant  density  level  was   53^  (see 
Figure  l).     The  53^  density  screen  was  then  divided  into  two 
screens,  three  screens,   and  four  screens  to  reduce  density 
level  by  varying  degrees.     Screens  were  divided  at  logical 
points  rather  than  according  to  specific  character  counts 
which  helped  maintain  a  uniform  density  level  across  the 
screens .     The  density  levols  for  the  multiple-screen 
displays  were  determined  by  averaging  the  density  of  each 
screen.     The  two-screen  display  had  an  average  density  level 
of   31^  per  screen,   the  three-screen  display  averaged  26^, 
and  the  four-screen  display  averaged  22S  (see  Figure  1). 
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The  screen  display  software  Included  a  management 
component  which  stored  the  data  collected  at  each  session  on 
disk  for  later  retrieval.     A  second  program  was  used  to 
provide  a  printout  of  each  subject's  responses,   reformat  the 
data  for  uploading  to  a  mainframe  for  later  anal/sis,  and 
add  the  data  to  an  archive  file  for  future  reference. 


Procedure 

From  2  to  15  subjects  attended  each  session  In  one  of 
two  computer  labs.     Both  labs  were  equipped  with  Apple  //e 
microcomputers  with  12  or  13  in.  monochrome  screens,  either 
one  or  two  5.25  In.   disk  drives,  and  64K  to  128K  of  memory. 
Proctors  began  the  session  with  a  brief  description  of  the 
purpose  of  the  study  after  which  they  booted  the  computers. 
The  first  screen  asked  subjects  for  their  name,   sex,  and 
status   Cgxaduete  or  undergraduate) .     Then,   subj  ects 
completed  the  9-ltem  attitude  survey  presented  on  the 
microcomputer .     Subs equent  scr eens  described  the  experiment 
and  explained  the  Information  contained  on  oach  screen. 

The  six  screen  comparisons  were  presented  in  a  random 
order.     The  density  level  randomly  selected  to  be  presented 
first  m  each  each  comparison  was   labeled  Design  tl  at  the 
top  of  the  screen  and  the  second  density  level  was  labeled 
Design  *2.     The  number  of  screens   In  the  design  and  the 
particular  screen  presently  being  viewed  was  Indicated  In 
the  lower  left  hand  corner  (e.g.,    "1  of   1",   "2  of  3",  etc.). 
A  prompt  In  the  lower  right  corner  of  the  screen  Indicated 
thdt  a  key  press  would  result  in  advancement  to  the  next 
frame.     After   viewing  both  designs,   subjects  had  the  option 
of  Indicating  their  preference  for  one  of  the  two  designs  or 
for  reviewing  either  or  both  designs .     Once  a  preference  was 
indicated,   presentation  of  the  next  pair  of  designs  was 
initiated.     This   process  was  then  repeated  for  each  of  the 
remaining  five  comparisons. 


Paired  Comparison  Selections 

Table  1  shows  the  proportion  of  subjects   (total  n  -  46) 
who  selected  each  density  level  when  paired  with  each  of  the 
alternative  Isvels .     These  proportions  reflect  a  curvilinear 
pattern,  with  preferences  tending  to  favor  the  two  middle 
density  levels  (eapecially  the  31%  level)  over  the  lowest 
(22H)  and  highest  (53'*)  levels.     Specifically,   the  31%  level 
was  favored  by  rhe  majority  of  subjects   (from  52  to  74 
percent)  over  each  of  the  other  three  levels:   the  26%  level 
was  favored  by  the  majority  (54  to  56  percent)  over  each  of 
the  two  extreme  levels . 


Insert  Figure  1  About  Here 


Results   and  Discussion 
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Insert  Table  1  about  here 


To  provide  an  overall  compare    m  of  the  density 
selection  rates,  a  scaling  procedure  derived  from 
Thur  s  t  one  s  mode 1  of  c  orapar  at 1 ve  j  udgraent  (see  Nunnally , 
1967;   Guilford,    1954)  was  used  to  represent  their  relative 
distances  on  an  Interval  scale.     The  procedure  Involves 
first  converting  the  proportion  values   (as   shown  in  Table  1) 
into  normal  curve  deviates-     For  exaroplo,   a  stimulus  that  is 
chosen  over  a  comparison  stimulus  by  84^  of  respondents 
would  have  a  normal  (z  score)  deviate  of  1.00,  representing 
the  area  in  the  distribution  that  is  1  standard  deviation 
above  the  mean.     The  normal  deviates  derived  for  each 
stimulus  are  then  averaged  to  produce  an  overall  mean.  To 
prevent  having  negative  values  on  the  final  scale,  the 
absolute  value  of  the  largest  negative  mean  is  added  to  each 
of  the  means.     Consequently,   the  "least  preferred"  stimulus 
on  the  final  scale  will  always   have  a  final  mean  value  of 
0.0.     For  the  present  preference  scale,   as  s hown  in  Figur e 
2,   the  scale  values  ranged  from  0.0  (22«6  density)  to  .49 
(31«6  density).     Although  the  26**  level  was  preferred  over 
the  53«»  level  in  their  direct  comparison  (see  above),  both 
had  identical  scale  scores  of    .19.     Baseu  on  these  overall 
scale  placements,   the  31^  level  can  be  considered  the  most 
frequently  preferred  and  the  22^  level  the  least  frequently 
pr  ef er r  ed . 


I ns er t  Figur e  2  about  her e 


To  verify  these  trends  statistically,    tabulations  were 
made  of  the  total  number   of  times  each  density  level  was 
chosen  by  subjects.     Because  euch  level  vjas  judged  on  three 
out  of  the  six  compArisons.   its  maximum  score  for  a  given 
subject  was   3.0.     Resultant  means  were  1.17,    1.46.    1.91,  and 
1.46  for  the  fou'.  density  levels,   respectively  (ordered  from 
lowest  to  highest  density).     The  density  selection  scores 
were  then  analyzed  in  two  ways.     First,  a  Friedman  ANOVA  by 
ranks,  a  nonpar ametr ic  test  (Hays,   1981),   was  used  to 
compare  their  ordinal  rankings  within  subjects.  Although 
this  test  is  less  powerful  relative  to  treating  the  scores 
as  interval  data  in  a  parametric  test,   it  was  considered 
less   likely  to  be  biased  by  the  built-in  interdependenc y 
between  individual  subjects'    four   selection  scores   (i.e..  if 
a  subject's  score  for   one  density  level  was  relatively  high, 
his/her  score  for  one  or  more  other  density  levels  would 
have  to  be  relatively  low  to  compensate).     Results  from  the 
^='riedman  test  were  signific  nt  ,   X^(3)  -  8.32,   £  <  .04. 
indicatirg  that  the  frequencies  with  which  the  density 
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levels  were  selected  were  different^     This   outcome  was  th^n 
substantiated  by  performing  a  repeated  measuies  multivariate 
analysis  of  variance  (MANOVA)  on  the  original  selection 
total  scores.   F  (3.43)  -  3.34.   £  <    .03.  Follow-up 
comparisons  of  m€»ans  were  reade  using  the  Tukey  HSD 
procedure.     Only  the  difference  between  the  31**  and  22^ 
levels  was  significant         <  .05). 

The  above  results   provide  information  on  how  the 
individual  density  levels  were  judged  relative  to  one 
another .     A  somewhat  different  question  concerns  whether  or 
not  overall  preferences  tended  to  favor,   as  the  literature 
suggests,   lower-density  over  higher-density  designs. 
However,   tabulations  across  subjects  on  the  six 
paired-comparison  trials  indicated  the  opposite  pattern;  156 
(57  percent)  selections  favored  the  higher  density  design 
whereas   only  120  (43^)  favored  the  lower  density  design. 
X^(i)  -  4.44.   £  <  .05. 
Individual  Differences  Outcomes 

Further  analyses  examined  density  preferences  and 
attitudes  as  a  function  of  subject  gender  and  academic  group 
(graduate  versus  undergradu«»ce) .     Dependent  variables  were 
the  four  density  level  total  scores,   the  total  number  of 
lower  density  designs   selected  across  trials .   and  scores  on 
each  of  the  nine  attitude  items.     Using  t_  tests  for 
indepen<?ent  samples,   none  of  the  group  effects  for  either 
individual  difference  variable  was  significant.  Finally, 
correlations  between  the  number  of  lower-density  designs 
selected  and  attitude  scores  were  consistently  low  and 
nonsignificant . 
Summary 

In  contrast  to  recommendations   in  the  literature 
(Allessi  St  Trollip.   1985:-  Bork.   1984.   1987;  Grabinger  .  1983; 
Hemes.    1984:   Hooper  St  Hannafin.    1986)  for  designing  lower 
density  screens,   these  results  showed  that  subjects  tended 
to  prefer  higher-density  screens.     The  relatively  stronger 
preferences  for  the  3iH  (intermediate)  density  level  may 
suggest   that  subj  ects  were  attempting  to  balance  aesthetic 
properties   (i.e..    perceived  readability  and  visual  appeal) 
with  either  or  both  (a)  the  degree  of  contextual  support  and 
(b)  the  number  of  screens   in  the  lesson.     If  the  latter  were 
the  key  factor,   then  preferences  for  the  lower  density  (more 
spacious)  designs  would  seem  likely  to  increase  if 
corresponding  increases  in  the  number  of  screens  were 
presented  by  presenting  only  the  first  screen  of  each  screen 
density  level  as  in  Grabinger' s   (1983)  study.     Study  II  was 
conducted  to  test  this  interpretation. 
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Study  II 


The  primary  Interest  In  Study  II  was  to  determine  the 
r epli cabl li t y  of  the  Study  I  results  when  orJ.y  the  first 
screen  of  each  density  level  was   presented.     It  was 
predicted  that  In  this  case,   stronger  preference  for  the 
lower  density  screens  would  be  Indicated  than  In  Study  I , 
since  reductions  In  density  level  would  not  require  having 
to  review  a  greater  number  of  frames. 


Subj  ects  and  Design 

Subjects  were  27  graduate  and  12  u.^.der  gr  aduat  e 
education  majors   (34  females  and  5  males)  who  volunteered  to 
participate  in  the  study  and  had  not  participated  In  Study 
I.     The  same  palred-compar Ison  design  as  In  Study  I  was 
employed . 

Materials  and  Procedures 

The  stimulus  materials  were  the  same  as  used  In  Study  I 
with  one  change.     Only  the  first   screen  for  each  density 
comparison  was  presented.     The  Instructions  were  modified  to 
indicate  that   subjects  would  view  only  the  first  screen  of 
information  in  the  six  designs,   but   in  a  real  lesson  they 
would  need  to  vi  ew  s  everal  screens  to  obtain  all  of  the 
relevant  information. 

As  in  Study  I.   the  first  screen  asked  subjects  for  the^ r 
name,   sex,   and  status   (graduate  or  undergraduate).  The 
9-item  attitude  survey  was  then  presented,    followed  by 
instructions  for  the  paired-comparison  task      The  six  screen 
comparisons  were  presented  in  a  random  order,   with  the 
density  levels  in  each  randomly  designated  as   Design  ^^1  or 
Design  t2  at   the  top  of  the  screen.     Again,   subjects  had  the 
option  of  viewing  either  or  both  designs  as  many  times  as 
desired  before  indicating  their  preference 


The  proportion  of  subjects  (total  n  -   39)  who  selected 
each  density  level  in  the  separate  comparisons  is   shown  in 
Table  II.     Here,   in  comparison  to  the  curvilinsar  trend  of 
Study  1.   the  pattern  is  directly  linear ,  with  the 
higher-density  design  consistently  preferred  over  the 
lower -densit y  design.     Application  of  the  linear  scaling 
procedure,   as  diagrammed  in  Figure  3.   reflects  this  pattern, 
showing  the  scaled  scores   to  increase,   from  0.0  to   .49,  as 
density  level  increases.     As  in  Study  1,   the  total  number  of 
times  subjects  selected  each  density  level  were  tabulated 
Overall  means  were  1,13.   1.49,   1.62.   and  1.77  (out  o£  a 
possible  3.0)  for  the  four  levels  respectively.  However, 
neither  the  Friedman  analysis   of   ordinal  rankings   nor  the 
repeated  measures  ANOVA  on  selection  total  scores  indicated 
a  significant  difference  between  levels,  although  the  latter 
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approached  significance  C  £  <    .08).     Across  all  comparisons, 
however ,   subj  ects  chose  the  higher -density  design  145  (62 
percent)  times  and  the  lower-density  design  only  89  (38 
percent).         (i)  m  12.93,   £  <    .001.     Thus,   compared  to  Study 
I.   while  no  particular  density  level  emerged  as 
significantly  more  or  less  desirable  than  others ,   there  was 
an  even  stronger  tendency  to  select  higher-density  design  of 
each  pair 


Individual  difference  comparisons  were  made  for  academic 
status,   but   not  for  gender  due  to  the  very  small  number  of 
males  (5  out  of   39)  in  the  sample.     Differences  were 
significant  on  the  selection  totals  for  two  density  levels. 
Undergraduates   selected  the  31%  level  an  average  of  1.08 
times   (i.e.,   on  36%  of   its  comparisons )  whereas  graduate 
students  selected  it  an  average  of   1.85  times  (a  60%  rate), 
t(38)  -  2.24,   £  <    .05.     For  the  53%  level,   the  opposite 
pattern  occurred,   with  the  undergraduate  students  selecting 
It  more  frequently  (  M  -  2.33.    rate  -  78%)  than  the  graduate 
students   (  M  -  1.51,   rate  -  50%),   t(38)  -  2.00,  £  <    .05.  No 
dif.  9rences   between  graduate  and  undergraduate  students  were 
f  ound  on  any  of  the  at  1 1 tude  items  or  on  the  total  number  of 
lower-density  designs   selected  across  trials. 


In  contrast  to  previous  studies  and  recommendations  in 
the  instructional  design  literature  (Allessi  &  Trolllp, 
1985:   Bork,    1984,    1987 Grabinger  ,^   1983;   Heines,  1984; 
Hooper  &  Hannafin,    1986),   subjects  in  the  two  studies 
indicated  a  strong  preference  for  learning  from  high  density 
screens  as   opposed  to  low-density  screens.     These  results 
were  generally  consistent  for  males  and  females,  and  for 
inexperienced  and  experienced  users.     The  suggestion  is  that 
the  use  of  realistic  stimulus  materials  may  produce 
different  results   than  obtained  with  nonrealistic  stimulus 
materials   (e.g.,  Grabinger ,    198  3)  or  with  informational 
(e.g.,  machine  status)  displays     (e  g.,   Danchak,  1976; 
Smith.   1980,   1981,  1982) 

A  question  still  remains  as  to  why  sub3ects  indicated  a 
preference  for  higher  density  screens   over  lower  density 
screens  in  the  individual  comparisons.      If  only  the  results 
from  Study  I  are  considered,   one  might  conclude  that  higher 
density  screens  were  selected  to  avoid  the  additional  effort 
(keypresses )  and  presentational  discontinuities  involved  in 
viewing  the  additional  screens   of   the  lower  density  version. 
In  Study  II,   however,     only  the  first   screen  of  each  density 
level  was  viewed,    yet   even  somewhat  stronger  preferences  for 
higher  density  screens  occurred.     Thus,   the  "additional 
effort"   hypothesis  suggested  from  Study  I  was  not  supported. 


Insert  Figure  3  About  Here 


Di  scus  s  ion 
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A  more  likely  inter pretflitlon  suggests  consideration  by 
subjects  of  the  informational  qualities  of  the  display. 
Figure  4  shows  two  screens  of  approximately  density, 
containing  realistic  content  and  the  other  nonreallstlc 
content ;   and  two  comparable  screens  of  approximately  53^ 
screen  density.     Seemingly,   in  visually  comparing  the  two 
nonreallstlc  or  "content  free"   displays,   the  lower  density 
screen  will  appear  more  spacious  and  easier  to  read.  When 
the  two  screens  containing  realistic  content .   however .  are 
compared,   one  must  not  only  consider  aesthetic  properties, 
but  also  the  amount  of  contextual  support  needed  to  learn. 
A  high  density  design  Increases  contextual  support  by 
presenting  maximum  Information  (both  main  Ideas  and 
supporting  explanations   or  examples)  on  a  single  frame.  By 
glancing  forward  or  backward  the  student  can  obtain  cues 
that  facilitate  the  processing  of  a  word  or  ohrase.  Low 
density  frames  minimize  this  contextual  support  which  should 
normally  disrupt  the  processing  of  Information.     (Consider , 
for  example,   the  extreme  case  of  reading  a  novel  in  which 
only  one  or  two  sentences  appear  on  each  page).     It  thus 
appears  that  the  contextual  properties  of  the  current 
displays   of  realistic  material  had  a  greater  Influence  on 
learner  preferences  than  the  aesthetic  properties.  Changing 
the  context  of  the  material  or  the  processing  demands  of  the 
task,   however,  might  alter  the  relative  importance  of  these 
two  features.     Further  research  is  needed  to  substantiate 
this  hypothesis 


Insert  Figure  4  About  Here 


The  present  research  calls  attention  to  two  salient 
problems  for  instructional  designers  and  researchers  in  the 
area  of  CBI  screen  displays.     First,   instructional  designers 
who  base  design  decisions   on  human  factors  research  should 
use  caution  when  attempting  to  apply  heuristics   proposed  for 
informational  displays  to  the  design  of  instructional 
displays .     Informational  displays .   which  are  designed  for 
"quick  glance"   reading,   present  Information  in  a  consistent 
location  and  vary  only  part        the  display  (e.g.,  monitor 
readouts).     Instructional  displays,   however,   are  designed 
for  slower  or  more  deliberate  processing  of  all  the  content 
Thus,   each  has  a  different  purpose  and  will  typically 
require  different  design  heuristics. 

Second,   for  the  reasons  proposed  in  the  preceding 
paragraph,   subjects  may  apply  different  perspectives  when 
reacting  to  nonreallstlc  as  opposed  to  realistic  stimulus 
materials  in  screen  design  studies.     Although  nonreallstlc 
materials  have  Internal  validity  advantages  for  bas  ic 
research,   results  need  to  be  verified  with  ecologically 
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valid  materials   before  heuristics  for  screen  designs  are 
generated  (Ross  &  Morrison.  1989). 

It  s hould  also  be  noted  that   sc reons  f or mat ted  In 
symbolic  notation  such  as  Twyman' s  (1981)  may  not  be 
directly  comparable  to  text  screens  of  the  same  computed 
text  density  level  due  their  use  of  solid  lines  of  x  s  or 
o's  as  contrasted  to  lines  of   nonsense  words  or  real  words 
separated  by  spaces .     It  1 s  recommended  that  rese&rchers 
Interested  In  content  free  stimulus  materials  consider  the 
potential  of  approximations  to  English  (Morrison.  1986; 
Shannon  ^  vieavor.   1964)  that  maintain  the  same  structure  as 
a  real^,8tlc  screen  without  conveying  meaning. 

As  a  final  point,   the  absence  of  an  operational 
definition  of  low  and  high  density  screens  makes  It 
dlf f Icult  to  compare  results  across  studies  and  to  trans  1 at o 
findings  Into  effective  design  practices.     To  provide  for 
consistency  In  design  and  research,   the  adoption  of  a 
standard  method  of  calculating  screen  density  Is  needod. 
Tunis'    (1983)  method  seems  appropriate  for  this  purpose  oy 
basing  screen  density  on  the  number  of  characters  and 
contiguous  spaces  on  the  screen.     Consistent  terminology 
should  also  be  used  In  classifying  and  referring  to  screens 
of  varying  density.     For  example,    screens  with  density 
levels  22'»  or  less  might  be  labeled  as  low-density,  those 
with  densities   between  26^  and  50^  as  medium  density,  and 
those  above  50%  as  high  density.     Although  these  cutoffs  are 
arbitrary,   they  approximate  discriminations  made  by  subjects 
In  the  present  research  and  would  help  to  eliminate  the 
current  situation  of  one  researcher's  "low-density"  display 
being  structurally  Identical  to  another's  high-density" 
display . 

It   IS  suggested  that  future  research  on  CBI  screen 
design  take  three  dlrectloxis .     First,   researchers  should 
focus  on  Identifying  optimum  screen  densities  as  opposed  to 
minimum  or  maximum  tolerable  densities.     This  approach 
differs  from  earlier  research  In  the  field  of  instructional 
technology  which  focused  on  such  factors  as  the  minimum  size 
for  projected  letters   (cf .    Phillips.    1976).     Based  on  the 
present  findings  regarding  learner  preferences .  the  optimum 
density  level  appears  to  be  between  31%  and  53%  (medium  to 
high).     Second,   additional  research  is  needed  to  test  the 
generality  of  these  findings  using  different  types  of 
stimulus  materials   (realistic   in  various  subject  areas  and 
levels  versus  nonrealistic ) ,     Quantitative  c  iented  subject 
material,  for  example,   may  require  different  design 
considerations  than  would  lessons  in  English  or  history. 
Although  the  present  results  were  similar  in  Studies   I  and 
II,   t»ie  use  of  multiple  frames  for  hjLgh  external  validity 
seems  advisable  to  pe*  .nit  generalization  of  findings  to 
actual  lessons.     Third,   current  research  on  CBI  screen 
design  has  focused  almost  exclusively  on  learner  preferences 
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for  different  designs.  Future  research  needs  to  Investigate 
the  Implications  of  these  designs  i.or  achievement  as  well. 
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Table  1 


Proportion  of  Times  Density  Levels  Within  Each  Paired 
Comparison  Were  Selected  in  Study  1 


Paired  Comparison 


22\  26H       22%  31H       22H  50H       26H  m       26H  50H       31%  50% 

.46     .54        .26     .74        .46     .54         35     .65        .56     .44        .52  .48 
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Affective  and  Cognitive  Influences  of  Textual  Display 

in  Printed  Instruction 


Abstract 

This  study  examined  the  influence  of  textual 
display  in  printed  instruction  on  the  attention, 
performance,  and  preference  of  90  pioseirvice  teachers, 
and  investigated  differential  effects  of  textual 
display  for  high  and  low  ability  learners. 

Textual  display  was  manipulated  through  three 
versions  of  printed  instruction.    Ability  was  measured 
by  GPA.    Attention  was  measured  by  the  Attention 
subscale  of  the  Instructional  Materials  Motivation  Scale 
(Keller,  1987) .    Performance  was  measured  by  an 
objective-referenced  test  of  recall.    Preference  was 
measured  by  items  from  the  IMMS  Attention  and  Confidence 
subscales. 

Data  were  analyzed  using  analysis  of  variance  and 
chi-square.    Results  indicated  that  there  was  an 
interaction  between  ability  and  textual  display  for 
performance  (p=.009),  but  not  for  attention.  Results 
also  indicated  that  textual  display  significantly 
influenced  performance  (p==*004),  but  not  attention.  The 
Moderate  Textual  Display  produced  significantly  higher 
performance  scores  for  low  ability  subjects  (p<.001). 
Results  of  the  test  for  preference  revealed  that,  of  the 
three  versions,  learners  preferred  the  text  exhibiting 
the  Moderate  Textual  Display. 
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Affective  and  Cognitive  Influences  of  Textual  Display 
in  Printed  Instruction 


Introduction 

Nationwide  surveys  show  that  over  50%  of  learning 
activities  are  based  on  the  use  of  textbooks  and  related 
print  materials,  and  that  in  some  areas,  teachers  rely 
almost  exclusively  prlnred  stimuli  (Bullough,  1988; 
Knirk  &  Gustafson,  1986;  Nelson,  Prosser,  &  Tucker,  1987) • 
Researchers  report  that  learners  tend  to  resist  print  more 
than  they  would  resist  a  lesson  from  some  other 
instructional  medixnn  (Knirk  &  Gustafson,  1986) •  Learners 
perceive  print  instruction  to  require  more  mental  effort 
than  other  delivery  systems,  and  this  perception 
influences  their  willingness  to  learn  from  textual  stimuli 
(Salomon,  1983)  .     Since  learner  interest  (or  lack  of 
interest)  in  the  instruction  can  influence  performance 
(Keller,  1983),  designers  must  consider  the  motivational  or 
affective  aspects  of  learning  from  text  when  developing 
print  instruction  (Hartley,  1987,  Sless,  1984;  Stewart, 
1989) . 

Keller's   (1983)   theory  of  motivation  illustrates  the 
interrelationships  of  motivation,  performance,  and 
instructional  influence  by  describing  the  effects  that 
learner  characteristics  and  environmental  factors  have  on 
motivation  and  performance.     Keller  (1986)  identified  four 
categories  of  learner  and/or  environmental  variables  which 
affect  motivation  to  learn:     attention,  relevance, 
confidence,  and  satisfaction.    Analysis  of  the  learner's 
motivational  status,  in  regards  to  these  four  requirements, 
determines  personal  input  deficiencies  which  might  be 
alleviated  by  environmental  inputs,  such  as  instructional 
stimuli,  to  promote  successful  performance. 

With  printed  instructional  stimuli,  a  personal 
input  deficiency  in  Attention  may  result  from  the 
instruction's  appearance.     This  deficiency  might  be 
alleviated  by  a  textual  display  (overall  visual 
appearance  of  printed  materials)  that  arouses  and 
maintains  the  learner's  interest.     This  interest  could 
lead  to  increased  motivation  to  learn  from  the  textual 
instruction,  which  might  positively  influence 
performance. 

The  primary  purpose  of  this  study  was  to  examine 
the  influence  of  textual  display  in  printed  instruction  on 
learner  attention  to  the  instruction  and  performance  of  the 
instructional  goal.     Secrndary  purposes  were  to  investigate 
any  differential  effects  of  textual  display  for  high  and 
low  ability  learners  and  to  determine  learner  preference 
for  textual  display. 
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Background 

Textual  display  has  been  defined  as  "the  manner  in 
which  text  information  is  presented  on  a  page  or  in  a 
chapter"   (Duchastel,  1982,  p.  167).     Grabinger's  (1985) 
study  of  textual  display  on  electronic  screens  revealed 
that  three  main  dimensions  affected  student  ratings  of 
textual  display:     organization,  structure,  and 
spaciousness.     The  importance  of  easily-perceived 
organization  and  structure  to  learning  from  printed  stimuli 
has  been  established  (Brandt,  1978?  Brooks  &  Dansereau, 
1983;  Gerrell  &  Mason,  1983?  Glynn  &  Britton,   1984;  O'Shea 
St  Sindelar,  1983;  Reder  &  Anderson^.  1982;  Rumelhart,  1980; 
Shimmerlik,  1978) . 

Spaciousness  has  also  been  found  to  influence  learning 
from  textual  instruction.     The  level  of  crowdedness  or 
spaciousness  can  be  referred  to  as  the  density  of  the 
textual  display,  which  is  differentiated  from  the  density  of 
the  textual  information.     Information  density  is  the  amount 
of  elaboration  provided  in  the  printed  instruction  (Fisher, 
Coyle,  &  Steinmetz,  1977;  Reder,  Charney,  &  Morgan,  1986). 
Though  the  two  types  of  d'^^nsity  were  not  differentiated  in 
their  study,  Morrison,  Ross,  and  O'Dell  (1988)  compared  high 
and  low  density  displays.     Results  revealed  no  significant 
differences  in  achievement,  though  high  density  displays 
were  preferred.     This  contradicts  Grabinger's  (1985)  results 
which  found  that  student?  prefer  text  designs  with  lots  of 
white  space  and  openness      Additional  research  could  help 
clarify  these  differences. 

Two  dependent  variables  that  might  be  examined  in 
relation  to  textual  display  are  attention  and  performance. 
Attention,  perception,  and  learning  are  intertwined  - 
practically  and  theoretically  (Fleming,  1987).  Motivation, 
including  the  arousal  and  maintenance  of  attention,  are 
affected  by  novel,  challenging,  moderately  complex  stimuli 
(Berlyne,   1966;  Deci,  1975;  Gagne,  1985;  Kagan,  1972;  ). 
Stewart  (198  8)   states  that  arousing  of  interest  and  focusing 
of  attention  can  be  induced  by  an  affective  aspect  of  text; 
and  Turnbull  and  Baird   (1975)   state  that  typographic 
techniques  can  be  employed  to  make  an  instuctional  text  more 
interesting  and  challenging.     Because  features  of  text 
design,  such  as  graphic  cues,  heading  placement,  and  layout 
(Bovy,  1981;  Brooks,  Dansereau,  Spurlin,  &  Holley,  1983; 
Coles  &  Foster,  Fleming  &  Levie,  1978;  Grabinger,  1985; 
Hartley,  1986;  Wager,  1987;  Thiagarajan,  1977;)   can  make 
printed  materials  novel,  challenging,  or  moderately  complex, 
textual  display  can  have  an  affect  on  learner  attention. 

Jonassen  (1982)  proposes  t^rt  cognitive  processing 
can  be  "text  induced."    Design      pects  of  the  text 
presentation  can  help  the  learner  semantically  encode  the 
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stimulus  and  conceive  linkages  between  features  of  the 
stimulus  that  will  later  serve  as  cues  for  retrieval 
(Wager,  1987),  so  the  textual  displ  y  can  be  manipulated  to 
facilitate  comprehension  (Duffy  &  Waller,  1985?  Stewart, 
1988).     Textual  organization  and  structure,  which  are 
logical  and  visually  explicit,  have  been  found  to  have  an 
effect  on  learning  (Brandt,  1978;  Cocklin,  Ward,  Chen,  & 
Juola,  1984;  Glynn  &  Britton,  1984;  Meyer,  1981,  1985; 
Singer,  1985;  Winn,  1981;  Witkin,  Moore,  Goodenough,  and 
Cox,  1977) .     The  structure  and  organization  of  printed 
instruction  can  be  made  explicit  in  text  (Stewart,  198  9b) 
through  verbal  and  typographic  cueing  systems   (Beck,  1984; 
Felker,  1980;  Glynn,  Britton,  &  Tillman.  1985;  Grabinger  & 
Albers,  1988;  Loman  &  Mayer,  1983;  Lorch  &  Chen,  1986). 
Textual  display  that  shows  explicit  structure  and 
organization  can  promote  performance. 

One  individual  difference  that  may  ha'  3  a 
moderating  effect  on  attention  and  perform  ice  is 
student  ability.     Learner  responses  to  textual  display 
could  be  based  on  different  capacities  for  processing 
information  (Jonassen,  198  2) .     Studies  have  shown  that  high 
and  low  ability  students  benefit  differentially  from 
typographical  cues  and  from  meaningfully  segmented  text 
(Beck,  1984;  Gerrell  &  Mason,  1983;  O'Shea  &  Sindelar, 
1983;  Wilson,  Pfister,  &  Fleury,  1981).     An  uncomplicated 
textual  display  that  explicitly  exhibits  the  organisation 
of  the  instruction  may  be  more  influential  with  lower 
ability  students.     The  ability  of  high  performance  students 
to  organize  as  they  read  and  to  use  the  cues  provided  may 
override  any  differences  for  textual  display. 


Method 

Independent  variables  were  textual  display  and 
ability.     Textual  display  refers  to  the  manner  in 
which  text  information  is  presented  on  a  page  and  was 
operationally  defined  as  the  arrangement  of  specified 
elements  on  the  printed  page  (Turnbull  and  Baird,  1975). 
This  variable  was  manipulated  by  varying  text  column 
width  and  position,  horizontal  and  vertical  spacing,  and 
heading  placement  in  three  versions  (Simple  Textual 
Display,  Moderate  Textual  Display,  Complex  Textual 
Display)  of  a  printed  self-instructional  text  (see  Table  1) . 
Entitled  The  Student  Teaching  Handbook,  this  text  presented 
information  on  the  expected  outcomes  and  criteria  for 
evaluation  of  the  student  teaching  experience.    Ability  was 
defined  as  the  subject* s  measured  achievement  and  motivation 
to  succeed  in  educational  activities   (Adams,  Waldrop, 
Justen,  &  McCrosky,  198  7)   and  was  measured  by  cumulative 
grade  point  average.     High  and  low  ability  subjects  were 
identified  as  those,  in  each  of  the  three  groups,  having  the 
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fifteen  top  and  bottom  CPAs,  respectively. 

Attention,  performance,  and  preference  were  the 
dependent  variables.    Attention  was  defined  as  the 
arousal  and  maintenance  of  interest  in  instruction  and 
was  measured  by  the  Attention  subscale  of  the 
Instructional  Materials  Motivation  Scale  (Keller,  1987). 
The  IMMS  is  a  36  item,  Likert-type  instrument  which  was 
developed  to  measure  the  presence  or  absence  of  the 
motivational  components  of  attention,  relevance, 
confidence,  and  satisfaction  in  instructional  materials. 
The  internal  consistency  for  the  instrument  is  .89  and 
the  reliability  of  the  Attention  Subscale  (12  items)  is  .88 
(Keller,  Subhiyah,  &  Price,  1989). 

Performance  was  defined  as  achievement  of  the 
instructional  goal  and  was  measured  by  a  14  item 
multiple-choice,  objective-re feranced  test  of  recall. 
Preference,  which  refers  to  learner  perception  of  textual 
display  across  various  descriptors,  was  operationally 
defined  as  learner  selection  of  one  of  the  three 
versions.    This  variable  was  measured  by  18  items  from  the 
IMMS  Attention  and  Confidence  subscales. 

The  data  were  analyzed  by  analysis  of  variance  and 
chi-square,  using  an  alpha  level  of  .05  for  all  tests. 
For  the  attention  and  performance  variables,  an 
interactive  model  was  followed.     Chi-square  was  used  to 
test  the  preference  variable. 

Subjects  were  90  students  enrolled  in  the  final 
field  experience  of  their  undergraduate  teacher 
education  program  at  a  small  public  college  in  South 
Georgia.    Student  teachers  were  randomly  assigned 
to  one  of  three  treatment  conditions  and  the  treatments 
and  testing  were  administered  during  the  Student 
Teaching  Orientation.     Each  group  received  instruction 
from  one  of  the  three  texts  and  were  then  given  the 
Attention  and  Performance  tests.    Upon  completion, 
copies  of  all  three  versions  were  distributed  and 
used  as  a  basis  for  response  to  the  Preference  measure. 
(A  total  of  128  student  teachers  were  assigned  and 
tested;  but  due  to  the  high  and  low  ability 
identification,  data  for  only  90  -  30  in  each  treatment 
condition  -  were  analyzed.) 


Results 

Results  indicated  that  there  was  an  interaction 
between  ability  and  textual  display  for  performance 
(p=.009),  but  not  for  attention  (see  Tables  2,  3,  &  4). 
Resvxlts  also  indicated  that  textual  display 
significantly  influenced  performance  (p=.004),  but  not 
attention  (see  Tables  3  &  4) .    The  Moderate  Textual 
Display  produced  significantly  higher  performance  scores 
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for  low  ability  subjects  (p<.001).     Results  of  the  test 
for  preference  revealed  that,  of  the  three  versions, 
learners  preferred  the  text  exhibiting  the  Moderate 
Textual  Display  (see  Tables  6  &  7) . 


Discussion 

Results  did  not  suport  the  expectation  that  textual 
display  would  significantly  influence  attention.  These 
findings  may  be  explained  by  the  situation  and  the 
subjects'  entry  level  of  motivation.     It  is  likely  that 
the  anticipation  of  the  student  teaching  experience 
induced  such  a  high  motivational  state  that  subjects  may 
have  needed  no  alleviation  of  attention  deficiencies  by 
the  instructional  stimuli. 

Analysis  showed  that  textual  display  had  a 
differential  effect  on  the  performance  of  high  and  low 
ability  learners.    These  results  show  that  a  moderate 
textual  display  can  improve  performance  of  low  ability 
learners. 

The  results  of  the  preference  test  show  that 
subjects  had  a  clear  prference  for  the  Moderate  Textual 
Display.     This  could  mean  that,  eventhough  they  may  have 
been  intrinsically  motivated  to  attend  and  perform,  the 
students  still  had  strong  feelings  about  the 
instructional  materials  they  would  prefer  to  receive. 
This  finding  suggests  thqt  there  are  affective  aspects 
of  textual  display  in  printed  instruction  and  that  text 
organization  and  structure  and  text  density  contribute 
to  these  affective  dimensions. 
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Table  1 

Variations  in  Textual  Display 


Textual  Display 


Element 

Simple 

Moderate 

Complex 

Text  column 

width 

medium 

medium 

wide 

position 

central 

side 

central 

Spacing 

vertical 

interlinear 

double 

single 

single 

paragraphs 

indented 

spaced 

indented 

horizontal 

right 

unjustified 

unjustified 

justified 

Headings 

intact 

marginal 

embedded 

Table  2 

Means   (and  Standard  Deviations)   for  Attention  and 
Performance  of  High  and  Low  Ability  Preservice  Teachers 

Textual  Display 


Simple      Moderate      Complex  Total 


Attention 


High  Ability 

35 

36 

37 

36 

(7.5) 

(6.7) 

(4.3) 

(6.2) 

Low  Ability 

36 

38 

36 

37 

(6.0) 

(6.6) 

(8.4) 

(6.9) 

Performance 

High  Ability 

88 

84 

87 

86 

(.05) 

(.08) 

(.07) 

(.07) 

Low  Ability 

77 

86 

79 

80 

(.11) 

(.06) 

(.10) 

(.10) 
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Table  3 

ANOVA  Sraromary  for  Performance 


Source 

SS 

df 

MS 

F 

P 

Explained 

.15 

5 

Textual  Display 

.01 

2 

.005 

.81 

.45 

Ability 

.07 

1 

.08 

12.43 

.001 

Interaction 

.06 

2 

.03 

5.04 

.009 

Residual 

.53 

83 

.006 

Total 

.68 

88 

Table  4 

Simple  Main  Effects  Analysis  of  Performance 


Source 

d£ 

MS 

F 

P 

Textual  Display 

Simple 

28 

.67 

4.49 

.001 

Moderate 

29 

.72 

1.97 

.441 

Complex 

28 

.69 

1.82 

.019 

Table  5 

ANOVA  Summary  for  Attention 


Source 

SS 

df 

MS 

F 

P 

Explained 

63.3 

5 

.34 

Textual  Display 

15.5 

2 

7.7 

.18 

Ability 

6.8 

1 

6.8 

.15 

.69 

Interaction 

41.2 

2 

20.6 

.47 

.63 

Residual 

3667.9 

83 

44.2 

Total 

3731.2 

88 
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Table  6 

Percentage  of  Learner  Selection  of  Textual  Display  and 
Chi-Square  Results 


ItGlflS 

Textual  Display 

Chi 

p 

Simple 

Moderate 

Complex 

Positive 

1 

41.4 

30.03 

.0000 

2 

25.2 

70  Q 

/  U  .  7 

73.32 

.  0000 

4 

32 

44.79 

.  0000 

5 

29.7 

fiA  1 

51.45 

.0000 

7 

25.8 

\J  y  m  'J 

4  7 

56.18 

.0000 

8 

38.3 

55.5 

e.3 

42.82 

.  0000 

11 

28.3 

66.1 

5.5 

38.59 

.0000 

12 

26.8 

67.7 

5.5 

52.01 

.0000 

13 

31.7 

35.7 

32.5 

.8262 

.9349 

14 

31 

63.5 

5.6 

49.41 

.0000 

17 

27.8 

67.5 

4.8 

52.76 

.0000 

18 

27.8 

66.7 

5.6 

63.56 

.ooco 

Negative 

3 

18.8 

4.7 

76.5 

9.86 

.0428 

6 

18 

9.4 

72.7 

9.03 

.0444 

9 

20.3 

3.1 

76.6 

10.31 

.0356 

10 

22.8 

5.5 

71.7 

24.22 

.0001 

15 

20.3 

3.1 

76.6 

10.31 

.03o6 

16 

13.3 

4.7 

82 

10.87 

.0376 

Table  7 

Summary  of  Distribution  of  Items  Measuring 
Learner  Preference  of  Textual  Display 

Textual  Display 


Items 

Simple 

Moderate 

Complex 

Positive 

First 

0 

11 

0 

Second 

11 

0 

0 

Third 

0 

0 

11 

Negative 

First 

0 

0 

Second 

6 

0 

0 

Third 

0 

6 

0 
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Foundations  of  the  Study 


Theoretical  Frauework 

Winn  (1989)  has  called  upon  instructional  designers  to 
"reason  from  basic  principles  of  learning  and  instruction 
rather  than  simply  following  design  models"  (p. 36).  This 
tact  requires  adherence  to  a  broader,  and  more  intricate, 
scheme  of  conceptual  considerations  than  has  been 
incorporated  into  the  typical  set  of  design  procedures 
advocated.    It  is  an  approach  that  demands  study  of  a  vide 
range  of  variables,  such  as  those  which  were  proposed  in  a 
framework  for  use  in  the  construction  of  a  comprehensive 
theory  of  instructional  design.    The  model  is  conceptual, 
rather  than  procedural  in  nature;  it  encompasses  clusters  of 
variables  related  to  the  learner,  the  content,  the  delivery, 
and  the  environment  (Richey,  1986).    The  validity  and 
utility  of  such  a  model,  however,  are  dependent  upon  data, 
preferably  data  which  has  been  replicated  in  various 
settings. 

The  present  study  highlights  the  role  of  environmental 
factors  in  the  learning  process.    Such  factors  have  been 
found  to  be  instrumental  in  learner  achievement  in  a  variety 
of  pre-adult  schooling  situations  (Brookover,  et.al.  cited 
in  Good  and  Brophy,  1986;  Anderson,  1982;  Randhawa  and  Fu, 
1973).    Walberg  (cited  in  Randhawa  and  Fu,  1973)  even  argues 
that  environmental  factors  play  the  most  important  role  in 
learner  achievement  next  to  general  aptitude.    The  research 
reported  here  represents  an  attempt  to  identify  similar 
effects  in  adult  training  situations. 

In  general,  few  critical  interactions  between 
environment  and  adult  learning  have  been  empirically 
demonstrated.    Cookson  (1986)  notes  that  "external  context 
variables  have  been  largely  ignored  in  the  adult  education 
literature"  (p.  133).    In  spite  of  this,  many  believe  in  the 
general  proposition  that  "learning  is  not  just  a 
psychological  process  that  happens  in  splendid  isolation 
from  the  world  in  which  the  learner  lives,  but  that  it  is 
intimately  related  to  that  world  and  affected  by  it" 
(Jarvis,  1987,  p.  11). 

Environment  has  been  described  in  terms  of  setting  and 
climate.    The  settings  (with  respect  to  instructional 
design)  are  structural,  describing  varying  education  and 
training  contexts.    In  this  study  the  setting  was  that  of 
industrial  training  in  a  large  manufacturing  company. 
Climate  relates  to  qualitative  variables.    Climate  can  be  a 
function  of  external  influences,  physical  materials  and 
arrangements,  the  participant  characteristics,  and  the 
atmosphere  of  the  organization  itself  (Richey,  1986).  Here, 
varying  types  of  organizational  climate  factors  are  ^ 
considered. 
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Research  Framework 


The  general  model  for  this  research  is  shown  in  Figure 
1.    This  model,  basically  an  input -process-output  model, 
shows  multiple  outcomes  of  training.     It  is  a  model  that 
includes  variables  which  are  hypothesized  to  influence 
effectiveness  of  the  instructional  design  process.     It  is  a 
model  which  suggests  causal  relationships.    Finally,        is  a 
model  that  suggests  transfer  of  training  effects. 

Training  Framework 

The  training  program  in  this  study  related  to 
industrial  safety    —  energy  control  and  power  lockout. 
(ECPL) .    The  topic  emerged  as  a  result  of  previous  research 
and  an  examination  of  company  accident  records.  The 
training  was  jointly  sponsored  by  a  major  automobile 
manufacturer  and  an  automotive  union  in  an  effort  to  change 
employee  attitudes  and  behaviors  in  relation  to  power 
lockout.    Locking  out  involves  shutting  down  the  assembly 
line  while  completing  diagnosis  and/or  repair  tasks. 
Failure  to  lock  out  has  ri>!sulted  in  serious  injury  and 
death.    The  locking  out  process,  however,  is  expensive  since 
it  completely  stops  production. 

The  ECPL  training  program  was  professionally  designed 
and  developed.    The  instructional  materials  consisted  of: 

a.  eight  modules, 

b.  a  videotape, 

c.  an  instructor's  manual 

d.  a  participant's  workbook,  and 

e.  a  portable  pocket  manual. 

The  first  module  in  the  series  was  a  leadership 
commitment  module  for  local  plants  and  union  management. 
The  remaining  seven  were  directed  towards  employees  who  are 
expected  to  follow  lockout  procedures  on  their  jobs.  They 
dealt  with  behavior  change  and  skill  development. 

Classes  were  conducted  at  each  manufacturing  plant  by 
nonprofessional  trainers  with  less  than  25  persons  in  a 
class.    The  trainers  presented  the  program  content,  led  the 
discussion,  and  demonstrated  the  ECPL  procedures.  Each 
training  period  consisted  of  seven  two  hour  sessions 
spanning  two  work  weeks;    the  courses  were  repeated  in  each 
plant  until  all  participating  personnel  had  attended. 

The  trainers  were  salaried  and  hourly  employees  whose 
duties  included  either  lockout  or  supervision  of  employees 
who  needed  to  lock  out.    The  trainers  received  four  days  of 
training  to  prepare  themselves  to  serve  as  instructors. 
They  were  on  temporary  assignment  until  the  program  was 
completed. 
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FIGURE  1 
A  MODEL  OF   AN  IKDUSTRIAL 
TRAINING    RESEARCH  EFFORT 
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Research  Design 

Desicrn  Summary.    The  research  was  conducted  within  the 
context  of  an  extensive  evaluation  of  the  safety  training 
program.    The  study  was  structured  as  a  pre-test  and  post- 
test  design.    Participants  completed  a  questionnaire  prior 
to  the  training  and  within  30-90  days  after  training-^ 
Current  knowledge,  behavior  patterns,  beliefs  and  attitudes 
were  determined;  in  addition,  standard  demographic  data  were 
collected. 

Dependent  Variables.    The  research  question  is  one  of 
determining  the  effects  of  entry  perceptions  of 
organizational  climate  factors  on  training  outcomes.  The 
training  outcomes  used  are  gain  scores  calculated  from  the 
pre-  and  post- test  measures  of: 

1.  Knowledge  (based  upon  performance  on  a  22  item 
ECPL  content  knowledge  test) , 

2.  Attitudes  towards  sa  "^ety  on  the  job  (based  upon 
self-report) , 

3.  On-the-job  application  of  general  safety 
precautions  (based  upon  self-report) ,  and 

4.  On-the-job  application  of  specific  behaviors 
taught  (based  upon  self-report  of  recent  use  of 
ECPL  procedures) . 

Independent  Variables.    The  organizational  climate 
factors  were  measured  by  employee  perceptions  of  20 
different  climate  elements.    These  measures  were 
replications,  to  a  great  extent,  of  those  used  in  previous 
industrial  research.    The  perceptions  were  measured  prior  to 
training.    The  variables  were  clustered  into  four  groups: 

General  Organizational  Climate  Factors 

a.  Cooperative  working  conditions 

b.  Involvement  in  decision  making 

c.  Physical  working  conditions 

d.  Ratings  of  overall  quality  of  work 

e.  Supervisor  ratings 

f.  Encouragement  to  be  innovative 

g.  Degree  to  which  working  conditions  promote 
productivity 

Organizational  Climate  Factors  Generally  Related 
to  Training 

a.  Union/company  support  safety 

b.  Supervisors  quick  correction  of  safety  hazards 
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c.  Extent  to  which  management  is  informed  of  safety 

■«f  W  V« 

d.  Pressure  from  colleagues  to  follow  safety  pro- 
cedures 

Organizational  Climate  Factors  Specifically  Related 
to  Training 

a.  Company/union  want  workers  to  lockout 

b.  Operation  more  important  than  lockout 

c.  Employees  accountable  for  not  locking  out 

d.  Physical  conditions  promote  easy  lockout 

e.  Lock  availability 

f .  Frequency  of  lock  use  into  next  shift  or  longer 

g.  Employee  authority  to  lockout 

h.  Work  makes  lockout  impossible 

i.  Lockout  not  a  common  practice 

Population  and  Sample 

There  were  two  trainee  populations  ~  hourly  personnel, 
and  salaried  and  supervisory  personnel*    The  samples  were 
selected  on  a  stratified  basis*    The  samples  came  from  five 
plants  representative  of  the  entire  company  in  terms  of 
plant  type,  size,  location  in  the  Metropolitan  Detroit  area, 
and  previous  involvement  in  research  projects.  All 
participating  plants  were  approved  by  representatives  of  the 
union  and  the  company  management.    There  were  over  5300 
employees  in  the  five  selected  plants,  and  a  total  of 
approximately  50,000  employees  were  trained  throughout  the 
corporation.    The  pre-test  sample  consisted  of  389 
employees,  with  284  of  these  retained  for  post-testing. 

Simple  random  selection  of  trainees  was  not  possible, 
but  random  selection  of  classes  was  permitted •    Ail  classes 
were  randomly  formed,  and  each  group  consisted  of  employees 
from  a  large  number  of  departments  within  the  plant.  The 
trainee  sample  represented  a  broad  spectrum  of  adult 
learners.    Table  1  describes  the  trainees  in  the  study  in 
terms  of  sex,  race,  age,  educational  level,  and  employment 
experience. 

Data  Analysis 

The  model  shown  in  Figure  2  provides  the  initial 
hypotheses  for  this  study.     (The  variables  in  this  model  are 
derived  from  the  general  r^.search  framework  presented  in 
Figure  1.)    The  model  suggests  that  attributes  of  the 
general  organizational  climate  determine  specific 
organizational  climate  characteristics;  and,  climate,  as  a 
whole,  effects  training  outcomes. 

Path  analysis  was  used  to  evaluate  and  to  estimate  the 
dimensions  of  this  model.    The  technique  enables  one  to 
estimate  the  causal  influence  of  a  number  of  variables 
considered  simultaneously.    The  initial  models  are 
formulated  on  the  basis  of  theoretical  expectations  and  then 


Table  1 

A  CttrparisOT  of  Hourly  and  Salaried 
SCPL  Trainee  Characteristics  Using 
Pra-Test  Data 


Charac±eristic 


Hourly 
N  % 


Salaried" 
N 


Ccmbined 
N 


Total  RespcixSents 

307 

100.0 

73 

100. 0 

380 

100.0 

Sex 

355 

93.4 

Male 

287 

93.5 

68 

93.2 

Fesnale 

20 

6.5 

5 

5.8 

25 

6.6 

Race 

312 

80.2 

VJhite 

250 

80.1 

1 0  Q 

Black 

43 

81.1 

10 

18.9 

53 

13.6 

Other 

14 

93.3 

1 

6.6 

15 

3.9 

66 

82,5 

13 

16.3 

80 

20.6 

122 

79,7 

31 

20.2 

153 

39.3 

46-55 

76 

75.2 

25 

24.8 

101 

26.0 

56  + 

46 

92.0 

A 

4 

12  9 

* 

62 

15.9 

61 

98.4 

1 

1.6 

Hloh  School 

99 

90.8 

10 

9.2 

109 

28.0 

vhrade  School/ 

171 

44.0 

Sane  College 

133 

77.8 

38 

22.2 

Oollege  Degree 

25 

73.5 

34 

or  nore 

9 

26.5 

8.7 

Years  on 

Presait  Job 

176 

45.2 

0-5 

139 

79.0 

37 

21.0 

6-10 

51 

89.5 

6 

10.5 

57 

14.7 

11-  15 

36 

75.0 

12 

25.0 

48 

12.3 

16-  20 

36 

90.0 

4 

10.0 

40- 

10.3 

21 

45 

75.0 

15 

25.0 

60 

15,4 

Years  at  company 

41.2 

17 

0-5 

10 

58.8 

7 

4.4 

6-10 

18 

69.2 

8 

30.8 

26 

6.7 

11-  15 

70 

84.3 

13 

15.7 

83 

21.3 

16-  20 

74 

90.2 

8 

9.8 

82 

21.1 

•  * 

137 

78.3 

38 

21.7 
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FIGURE  2 

A  MODEL  OF  THE  HYPOTHESIZED  RELATIONSHIPS 
BETWEEN  ORGANIZATIONAL  CLIMATE  VARIABLES 
AND  TRAINING  OUTCOMES 
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evaluated  empirically.    McDonald  ana  Elias  have  recommended 
the  use  of  path  analysis  in  relating  teaching  performance 
data  to  student  achievement,  proposing  such  techniques  as 
tools  in  building  instructional  theory  (McDonald  and  Elias, 
1976  cited  in  Gage,  1977). 


Results 

The  results  support  the  general  notion  that 
organizational  climate  variables  influence  training 
outcomes.    However,  only  one  of  the  four  models  constructed 
yield  results  that  are  worthy  of  extensive  comment.  This 
was  the  model  showing  the  role  of  organizational  climate 
variables  in  relation  to  on-the-job  behavior  directly 
related  to  the  training  content.    The  other  three  models 
(relating  to  general  on-the-job  behavior,  attitude  change, 
and  knowledge  retention)  indicate  that  organizational 
climate  has  only  a  very  small  role. 

A  Model  Relating  Organizational  Climate  Factors  to  Specific 
On-the-Job  Behavior  After  Training 

The  path  diagram  shown  in  Figure  3  is  the  basis  of  this 
discussion.    Supporting  data  is  displayed  in  Table  2. 

The  model  shows  a  complex  network  of  interrelated 
organizational  climate  factors  which  explain  nearly  20%  of 
the  variance  in  the  training  outcomes;  it  supports  the 
hypothesized  pattern  of  general  factors  predicting  the  more 
specific  factors,  with  the  major  effects  produced  by  the 
most  specific  climate  measures    —    those  directly  related 
to  the  training  content.    The  direct  causal  influences  of 
recent  on-the-job  lockout  behavior  relate  to  employee 
perceptions  of  management  actions,  and  to  the  extent  to 
which  employees  feel  they  are  empowered  to  be  involved  in 
plant  operations.    An  examination  of  those  variables  which 
have  both  direct  and  indirect  relationships  to  training- 
specific  on-the-job  behavior  not  only  shows  the  continued 
influence  of  these  two  types  of  variables,  but  also  of  the 
influence  of  colleagues  and  physical  working  conditions. 

Employee  Perceptions  of  Management  Actions.    There  are 
five  variables  in  the  model  related  to  how  the  employees 
perceive  management  —    lockout  use  into  next  shift, 
operation  more  important  than  lockout,  union/company  support 
safety,  supervisor's  quick  correction  of  safety  hazards,  and 
the  general  rating  of  the  supervisor's  performance.  The 
first  four  speak  to  management  credibility  when  it  comes  to 
safety  and  lockout.    Do  management  actions  really 
demonstrate  that  supervisors  think  safety  matters? 

These  variables  concerned  with  management  are 
interrelated.    Employees  who  think  supervisors  really  are 
more  concerned  with  production  "when  push  comes  to  shove" 
are  less  likely  to  think  the  company  and  the  union  really 
support  safety  and  are  less  likely  to  think  their 
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FIGURE  3 

A  MODEL  OF  THE  EFFECTS  OF  ORGANIZATIONAL 
CLIMATE  FACTORS  ON  SPECIFIC  ON-THE-JOB  BEHAVIOR 
OUTCOMES  OF  TRAINING* 
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Table  2 

Hypothesized  Organizational  Climate 
Causes  of  Specific  On-The-Job 
Behavior  Outcomes  of  Training 


Endogenous 
Variable 

Causal 
Variable 

B 

SE  B 

T 

Prob. 

Constant 

3.105 

.249 

Lockout 
Use  Into 
Next  Shift 

Supv.  Rating  .196 

Colleague  Pressure. 188 
to  be  Safe 

.064 
.086 

3.052 
2.202 

.002 
.028 

Mean  =  3.977 
SD  =  1.740 

R  Square  =  .38 

N 

=  389 

Operation 
More  Important 
Than  Lockout 


Mean  =  2.771 
SD   =  1.080 


Constant 4.156 


Supv.  Correction  -.421 
of  Safety  Hazards 

Union/Company  -.155 
Support  Safety 


175" 


.065 


,070 


R  Square  =  .117 


N  =  389 


■6.434 
■2.244 


,000+ 
,025 


Employee 
Authority 
To  Lockout 


Mean 
S.D. 


1.874 
1.080 


Constant  17206 

Union/Company  .341 
Support  Lockout 


.106 
.048 


R  Square  =  .117         N  =  389 


Colleague  Pressure 
To  Be  Safe 

Mean  =  2.219 
S.D.  =  1.018 


Constant 

Encouraged 
To  Innovate 


1.999 
.093 


,101 
,037 


R  Square  =  .016 


N  =  389 


,M78 


,000+ 


2.514 


.012 


ERIC 


10  453 
.  459 


Table  2  Cont, 


Endogenous 

Variable 

Prob 


Union /Company 
Support  Safety 


Mean  =  1.959 
SD    =  1.086 


Supervisor 
Correction  of 
Safety  Hazards 


Mean  =  2.563 
SD    =  1.157 


Casual 
Variable 


B 


SE  B 


Constant  1.145 

Involvement  in  .133 
Decision  Making 

Encouraged  to 

Innovate  .146 

Physical  Working 
Conditions  .134 

Cooperative  Working 
Conditions  -.099 


R  Square  =  .160 


.122 
.041 

.051 
.048 
.050 
389 


Constant  1.892 

Encouraged  to 

Innovate  .198 

Physical  Working 
Conditions  .147 

Cooperative  Working 
Conditions  -.194 


Supervisor 
Ratings 


.107 


.133 
.052 
.051 
.056 
.049 


3.231 

2.867 
2.C12 
-1.985 


3.787 
2.878 
-3.454 

2.194 


.001 

.004 
.005 
.048 


.000+ 
.004 
.000+ 
.029 


R  Square  =  .125 


N  =  389 
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Table  2  Cont. 


Dependent 
Variable 


Casual 
Variable 


SE  B 
.546 


Prob 


Gains  in  Recent 
Job  Lockout 
Behavior 


Constant 

Lockout  Use 
Into  Next 
Shift 

Involvement 
in  Decision 
Making 


17923 


.462 


.346 


Employee  Authority 
to  Lockout  .323 

Operation  More 
Important  than 
Lockout  -.194 


,084 


.096 


,137 


,101 


5.500 


3.591 


2.354 


■1.925 


,000+ 


,000+ 


,019 


,055 


Mean  =  .884 
SO   =  2.684 


R  Square  =  .194 


N  =  284 
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supervisors  correct  safety  hazards  when  they  are  pointed 
cut.    Correspondingly /  a  belief  that  there  is  general 
support  of  safety  relates  to  the  belief  that  employees  are 
given  the  authority  to  lockout • 

The  fifth  significant  variable  in  this  category  is  a 
more  general  rating  of  the  supervisor.    This  is  the  only 
measure  in  this  grouping  not  related  to  the  topic  of  the 
training;  but,  this  partially  explains  two  of  the  other  more 
training-specific  measures  in  this  group  —  lockout  use  into 
the  next  shift  and  supervisor  correction  of  safety  hazards. 
Supervisors  received  higher  ratings  if  employees  felt  that 
the  supervisors  1)  quickly  corrected  hazards  which  were 
brought  to  their  attention,  and  2)  shut  down  production  — 
for  a  long  time,  if  necessary  —  by  locking  out  to  insure 
safety. 

Employee  Empowerment.    The  next  most  influential  group 
of  variables  related  to  the  extent  to  which  employees  were 
given  rights  and  responsibilities.    These  factors  were  the 
extent  to  which  trainees  felt  they  had  the  authority  and^ 
responsibility  to  lockout,  were  involved  in  decision  making 
in  the  plant,  and  were  encouraged  to  devise  new  and  better 
ways  of  doing  things  in  the  plant.    Two  of  the  three  factors 
are  general  climate  issues,  not  related  to  the  training 
content.    The  importance  of  this  group  of  variables  is  shov/n 
by  the  fact  that  two  of  the  three  measures  in  the  group 
directly  influence  employee  lockout  behavior. 

Colleqial  Relationships,    Variables  relating  to  the 
influence  of  other  workerjs  also  play  a  role  in  this  model. 
If  workers  not  following  safe  practices  generally  irritate 
others,  even  if  no  one  gets  hurt,  there  is  a  tendency  for 
supervisors  to  permit  use  of  lockout  techniques  into  th^ 
next  shift. 

The  effects  of  a  ge;)eral  cooperative  atmosphere  among 
workers  shows  a  different  pattern;  these  data  show  increased 
collegiality  relates  to  a  more  negative  perception  of 
supervisors'  safety  attitudes  and  behaviors.    This  finding 
is  difficult  to  explain.    Perhaps  there  is  a  bonding  among 
workers  which  leads  to  increased  cooperation  when  employees 
feel  there  is  not  a  genuine  support  of  safety  in  the  plant. 

Phvsical  Working  Conditions.    Finally,  the  degree  of 
satisfaction  one  has  with  the  physical  working  conditions 
does  have  an  indirect  role  in  predicting  on-the-job 
behaviors  effects  of  training.    Better  physical  conditions, 
as  might  be  expected,  predict  more  positive  perceptions  of 
climate  factors  generally  related  to  the  training  content. 

Other  Models  of  Organizational  Climate  Influences  on 
Training  Outcomes 

The  data  analyses  with  respect  to  the  other  dependent 
variables  —  general  on-the-job  behavior  outcomes,  general 
attitude  outcomes,  and  knowledge  ^^etention  —  also  show  that 
organizational  climate  variables  play  a  role  in  explaining 
training  outcomes;  however,  it  is  so  small  that  little 


discussion  is  warranted.     (See  Figures  4,  5,  and  6  and  the 


to  specific  behavioral  outcomes,  organizational  climate 
factors  seem  to  have  the  most  influence  on  general  attitude 
changes,  accounting  for  nearly  eight  per  cent  of  the 
variance.    There  are  negligible  effects  in  the  other  models. 

Perhaps  one  result  that  should  be  Moted  is  that  the 
climate  variables  specifically  related  to  training  tend  to 
disappear  when  the  training  outcome  being  considered  is 
general  in  nature. 


The  major  conclusion  of  this  study  is  that  if  one  is 
intent  on  designing  an  effective  industrial  t.:aining 
program,  consideration  needs  to  be  given  to  more  than 
systematic  procedures  for  designing  programs,  materials  and 
media.    One  must  consider  all  factors  that  affect  the 
learning  process.     In  an  adult  industrial  training  program, 
in  particular,  where  the  goal  is  not  simply  knowledge 
acquisition,  but  knowledge  application,  dimensions  of  the 
larger  system  must  be  taken  in  account. 

On-the-job  application  of  training  content  involves 
elements  of  transfer  of  training,  elements  of  motivation, 
and  elements  of  information  retrieval.    What  is  the  nature 
of  the  role  of  organizational  climate  in  all  of  these 
processes?    Inherent  in  one's  perception  of  an  organization 
is  the  picture  of  the  extent  to  which  the  organization  (both 
colleagues  and  supervisory  personnel)  value  the  behaviors 
which  are  being  promoted.    These  values  are  typically 
inferred  through  daily  performance,  or  modeling.  Another 
major  factor  suggested  by  the  results  of  the  present  study 
is  the  perceived  influence  of  the  individual  within  the 
organization.     Both  modeling  and  perceived  influence  appear 
to  have  a  substantial  bearing  on  the  application  of  training 
content.    The  physical  working  conditions  have  far  less^ 
impact  on  behavior  changes  which  occur  after  a  systematic 
training  experience. 

Behavior  Modeling  Effects 

Killen  and  Robinson  (1988)  have  cited  the  major 
influence  of  "social-environmental  modeling"  (p.  175) , 
especially  in  situations  related  to  health  behaviors.  Their 
conclusions  were  based  primarily  on  research  relating  to 
children;  the  present  study  suggests  that  modeling  is 
important  for  adults,  as  well.    Transfer  of  knowledge  to 
workplace  practice  was  clearly  promoted  by  general  adherence 
to  safety  principles  by  both  colleagues  and  supervisory 
personnel,  as  well  as  specific  demonstration  of  lockout 
procedures  by  supervisors.  Overall,  the  most  influence  comes 
from  supervisors  rather  than  co-workers. 

These  behavior  patterns  are  consistent  with  The  Health 
Belief  Model  which  explains  health  behavior  in  terms  of 
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one's  perceptions  of  1)  the  severity  of  a  condition,  2)  the 
benefits  of  the  preventive  action  minus  the  barriers  to  the 
action,  and  3)  the  presence  of  action  cues  (Maiman  and 
Becker,  1974,  as  cited  in  Murphy,  1989).     Each  of  these 
characteristics  can  be  explained  in  terms  of  modeling 
behaviors  within  the  organization,  but  especially  by 
management.     If  one's  supervisor  locks  out,  the  message  is 
sent  that  the  lockout  benefits  arc  greater  than  the 
inconveniences  it  causes.    Seeing  one's  supervisor  lock  out 
may  also  sei-ve  as  a  reminder,  a  cue  to  action. 
Modeling  behaviors  among  co-workers  are  also  important, 
since  they  create  a  climate  in  which  the  supervisors  tend  to 
enforce  the  lockout  procedures  (n.b.  "colleague  pressure  to 
be  safe"  leads  to  "lockout  use  into  next  shift") . 

Role  of  the  Individual  within  the  Organization 

The  mere  presence  of  model  behavior  is  insufficient  in 
itself  to  promote  on-the-job  application  of  the  training 
content.    The  modeling  provides  extexral  stimulation  to 
apply  the  content  of  the  training,  but  the  trainee  must  also 
perceive  that  he  or  she  has  a  role  of  some  influence  in 
plant  operations.    These  attitudes  apparently  provide  at 
least  part  of  the  internal  motivation  necessary  to  follow 
the  prescribed  safety  procedures.    Such  results  are 
interesting  in  light  of  the  energies  being  placed  upon 
employee  involvement  and  union-management  cooperation  in 
many  companies  as  an  avenue  towards  improved  production 
quality. 

Health  and  safety  topics  often  involve  behavior  ir> 
which  knowledr>e  of  correct  procedures  is  not  the  issue. 
Rather,  the  problem  is  one  of  forming  habits  of  known 
procedures.    Even  when  survival  issues  are  at  stake  (i.e. 
serious  injury) ,  habits  are  not  forthcoming  without  internal 
beliefs  that  provide  a  motivating  basis  for  action.  The 
present  study  supports  this  conclusion;  there  was  a 
correlation  between  a  number  of  attitudinal  factors  and  on- 
the-job  behavior,  but  not  between  knowledge  of  procedures 
and  their  use.   (Knowledge  retention  showed  no  correlation  in 
the  preliminary  analysis  with  the  on-the-job  behavior 
dependent  variable  and  was  dropped  from  the  model.) 

Stimulating  New  On-the-Job  Behavior 

Industrial  safety  behavior  changes,  therefore,  seem 
influenced  not  so  much  by  knowledge  but  by  a  climate  in 
which  supervisors  and  co-workers  value  the  desired  actions 
and  by  the  perceptions  that  employees  are  valued  themselves 
in  the  organization.    As  previously  noted,  the  Health  Belief 
Model  also  indicates  the  need  for  a  cue  to  action;  the 
modeling  behavior  of  others  was  suggested  as  a  possible 
environmental  stimulus.    Other  situations  can  also  serve 
this  function.     Fox  example,  if  an  employee  is  injured  on 
the  job,  this  could  also  serve  as  a  cue  to  action.    The  cues 
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can  be  planned.     In  the  context  of  this  study,  posters  were 
placed  in  the  plants  as  reminders  to  lock  cut. 

The  need  for  specific  action  stimuli  is  also  supported 
by  previous  research  and  theory  of  human  information 
processing  and  transfer  of  training.    Some  cue  is  required 
tj  facilitate  information  retrieval.    The  cue  is  typically 
embedded  in  the  instruction,  and  consaquently  one's  long- 
term  memory.    Reminders  are  needed  to  retrieve  the 
knowledge,  in  this  instance  the  lockout  procedures,  and  use 
it  in  the  proper  manner.    Repetition  of  this  retrieval-use 
process  then  promotes  automat icity,  or  habit.    Such  habits, 
however,  are  dependent  first  on  having  the  correct 
information  stored  and  working  in  an  environment  which  is 
supportive  of  attending  to  the  cue. 

Implications  for  Instructional  Design 

Well-designed  materials,  courses,  and  programs 
influence  knowledge  acquisition  and  retention.    If  knowledge 
is  only  an  intermediate  goal  and  the  ultimate  aim  is  to 
change  behavior,  then  the  designer  must  be  guided  by  a  model 
more  complex  than  the  typical  instructional  systems  design 
framework.    The  new  model  must  encompass  the  nature  of  the 
organization  in  which  the  behaviors  are  to  be  exhibited. 
This  present  study  suggests  that  the  isore  important 
organizational  characteristics  relate  to  social- 
psychological  dimensions  rather  than  physical  properties. 

Design  procedures  should  be  expanded,  as  well. 
Specifically,  this  study  has  implications  for  design  aspects 
such  as  the  needs  assessment,  the  types  of  examples 
incorporated  into  instructional  materials,  and  the  follow- 
up.    The  most  useful  needs  assessment  should  include  more 
than  trainee  needs  and  organizational  technical  needs. 
Employee  perceptions  of  the  work  environment  and  their  roles 
within  that  context  should  be  determined.  Instructional^ 
materials  should  be  constructed  to  include  those  cues  which 
one  expects  to  '^.timulate  the  desired  performances.  For 
example,  in  this  case  videos  could  emphasize  the  average 
worker  locking  out,  and  include  situations  in  which  lockout 
is  enforced  by  the  supervisor  and  plant  management.  These 
cues  are  obvious?    however,  other  cues  could  also  be 
identified  and  portrayed. 

Finally,  designers  can  make  efforts  to  promote 
organizational  support  of  the  desired  behavior.    They  can 
recommend  that  supervisors  and  plant  management  be  included 
in  the  training,  and  suggest  periodic  follow-up  instruction 
for  everyone,     in  addition,  follow-up  studies  should  assess 
not  only  the  continued  use  of  the  training  content,  but  the 
organizational  climate  most  directly  related  to  the 
training.    In  summary,  these  suggestions  provide  support  for 
the  integration  of  both  the  cognitive  and  affective  domains 
into  complex  instructional  sequences,  thus  si^porting  the 
work  of  Martin  and  Briggs  (1986) . 
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Implications  for  Adult  Learning 


The  findings  of  this  study  speak  not  only  to  design 
practice,  but  also  to  principles  of  adult  learning,  and  how 
it  differs  from  learning  of  pre-adults.    Adult  learning  is 
usually  work-related.    The  National  Center  for  Education 
Statistics  found  that  almost  49%  of  adult  education  content 
in  the  United  States  is  concerned  with  one's  work  (cited  in 
Darkenwald  and  Merriam,  1982).    This  present  study  indicates 
that  one's  perceptions  of  his  or  her  workplace  accounts  for 
almost  20  percent  of  the  variance  in  the  behavioral  effects 
of  work-related  training.    This  is  reminiscent  of  Bloom's 
(1976)  finding  from  studies  of  children  that  "affect  toward 
the  school...  will  account  for  as  much  as  20  percent  of  the 
variation  in  school  achievement"  (p.  92) . 

Bloom's  findings  were  minimal  in  relation  to  children 
in  grades  1-5  and  more  pronounced  for  subjects  in  junior  and 
senior  high  school.    He  concludes  that  attitudes  towards 
school  become  more  influential  as  one's  experience  in  the 
school  setting  increases.    At  the  adult  level,  the 
perceptions  of  one's  environment  continue  to  be  influential. 

Bloom's  pre-adult  findings  and  the  adult  findings  in 
this  study  differ  in  the  nature  of  the  dependent  variable. 
Bloom's  findings  concern  cognitive  academic  achievement 
reflected  in  grade  point  averages  and  composite  scores  on 
achievement  tests,  measures  generally  comparable  to  the 
knowledge  retention  scores  in  this  study.    Here  the 
organizational  climate  variables  are  influential  primarily 
in  relation  to  on-the-job  behavior,  an  indication  of 
transfer  of  training. 

For  adults,  the  transfer  task  seems  to  encompass  a 
wider  range  of  predictor  variables  than  the  knowledge  recall 
task.    While  transfer  of  training  is  viewed  primarily  from 
the  theory  of  common  elements  (Butterfield  and  Nelson, 
1989),  this  study  proposes  the  addition  of  other  factors  in 
explaining  the  transfer  process.    The  organizational  climate 
factors  seem  to  provide  a  motivation  to  use  the  learned 
procedures.    Transfer  appears  to  be  dependent  upon  not  only 
capability,  but  inclination. 

Summary  and  Next  Steps 

Organizational  climate  and  its  role  in  adult  learning 
is  often  discussed,  but  the  discussions  have  produced  little 
data  to  support  opinions.    This  Study  has  identified  a  role 
organizational  climate  plays  in  industrial  training;  it  is 
not  known  if  the  pattern  is  consistent  for  other  adult 
learning  situations,  both  formal  and  informal.    Not  only 
could  the  setting  in  this  study  produce  unique  findings,  but 
the  findings  may  be  influenced  by  the  fact  that  the  study 
was  concerned  with  required  training  on  a  topic  with  high 
risk  implications.    The  population  involved,  however,  was  a 
com.prehensive  group  with  a  broad  range  of  demographic 
characteristics;  it  was  not  an  exceptional  group  of  adults. 
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Figure  4 

A  MODEL  OF  THE  EFFECTS  OF  ORGANIZATIONAL  CLIMATE  FACTORS 
ON  GENERAL  ON-THE  JOB  BEHAVIOR  OUTCOMES  OF  TRAINING  * 


*PATH  COEFFICIENTS  ARE  INDICATED  ON  EACH  PATH 


Table  3 

Hypothesized  Organizational  Climate 
Causes  of  General  On-The-Job  Behavior 
Outcomes  of  Training 


Endogenous 
Variable 


Supervisor 
Correction  of 
Safety  Hazards 


Mean  =  2.563 
SD     =  1.157 


Casual 
Variable 


Constant 

Encouraged  to 
Innovate 


B 


SE  B 


.198 


Physical  Working 
Conditions  .147 


Cooperative 
Working  Cond. 


-.194 


Supervisor 
Rating 


.107 

R  Square  =  .125 


.052 
.051 

.056 

.049 

N  =  389 


Prob 


3,787 
2.878 

-3.454 
2.194 


.000+ 
.004 

.000+ 
.029 


Dependent 
Variable 

Casual 
Variable 

B 

SE  B 

T 

Prob 

Constant 

.445 

.267 

Gains  in 
On-the-job 
Safety 
Behavior 

Supervisor 
Correction  of 
Safety  Hazards 

Overall  Quality 
of  Work 

.202 
.139 

.087 
.073 

2.336 
1.911 

.020 
.057 

Mean  =  .356 
SD   =  1.701 

R  Square  = 

.035 

N  =  284 
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FIGURE  5 

A  MODEL  OF  THE  EFFECTS  OF  ORGANIZATIONAL  CLIMATE  FACTORS 
ON  GENERAL  APTITUDINAL  OUTCOMES  OF  TRAINING  * 


GEN.  ORG.  ^    ORG.  CLIMATE  FACTORS  ^ORG.  CLIMATE  FACTORS  ^TRAINING 

CLIMATE  GENERALLY  RELATED  SPECIFICALLY  RELATED  OUTCOMES 

FACTORS  TO  TRAINING  TO  TRAINING 


PHYSICAL  WORKING 
CONDITIONS 


INVOLVEMENT  IN  DECISION 
MAKING 


*  PATH  COEFFICIENTS  ARE  INDICATED  ON  EACH  PATH 
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Endogenous 
Variable 


Physical 
Conditions 
Promote 
Easy  Lockout 


Mean  =  2.213 
SD    =  1.025 


Supervisor 
Correction  of 
Safety 
Hazards 


Table  4 

Hypothesized  Organizational  Climate  Causes 
of  General  Attitudinal  Outcomes  of  Training 


"Casual 
Variable 


Constant  171^:4 

Union/Company 

Support  Safety  .233 

Supervisors 

Correction  of 

Safety  Hazard  .172 


Conditions  Promote 
Productivity  .077 


R  Square  =  . 171 


Mean  =  2.563 
SD    =  1.157 


Constant 1.892 

Encouraged  to 
Innovate  .198 

Physical 
Working 

Conditions  .147 

Cooperative 
Working 

Conditions  -.194 
Supervisor 

Rating  .107 


R  Square  =  .125 


SE  B 
.138 


,050 


.046 


.035 

n  =  389 


.133 
.052 

.051 

.056 
.049 

N  =  389 


4.681 


3.722 


2.228 


3.787 

3.878 

-3.454 
2.194 


Prob 


.000+ 


,000+ 


.026 


.000+ 

.004 

.000+ 
.029 
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Endogenous 
Variable 


Union/Company 
Support  Safety 


Mean 
SD  = 


1.959 
1.086 


Casual 
Variable 


SE  B 


Constant  1.145 

Involwement 
in  Decision 

Making  .133 

Encouraged  to 
Innovate  .146 

Physical  Working 
Conditions  .134 

Cooperative 
Working 

Conditions  -.099 
R  Square  =  .160 


.122 

.041 
.051 
.048 

.050 
N  =389 


3.231 
2.867 
2.812 

-1.985 


Prob 


.001 
.004 
.005 

.048 


dependent 
Variable 


Casual 
Variable 


B 


SE  B 


Prob 


Gains  in  General 
Attitudes 
Toward  Safety 


Mean  =  .102 
SD    =  1.053 


Constant .577 

Union/Company 
Support  Safety  .184 


Physical  Conditions 

Promote  Easy 

Lockout  .144 


R  Square  =  .075 


.155 
.060 


.063 
N  -  284 


3.089 


2.275 


.002 


.024 
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FIGURE  6 

A  MODEL  OF  THE  EFFECTS  OF  ORGANIZATIONAL  CLIMATE 
FACTORS  ON  KNOWLEDGE  RETANTION  OUTCOMES  OF  TRAINING 
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Table  5 

Hypothesized  Organizationa"!  Cliniate 
Causes  of  Knowledge  Retained  After  Training 


Endogenous 
Variable 

Casual 

Variable  B 

SE  B 

T 

Prob 

Constant  1.342 

.142 

Employees 
Accountable 
for  Not 
Locking  Out 

Col  league 
Pressure  uO 

be  Safe  .237 

.051 

4.662 

.000+ 

Mean  =  2.116 
SD     =  1.060 

Overall  Quality 

Of  Work  .121 

R  oPuare  =  .Uoi 

.037 

M  -"300 

3.241 

.001 

Constant  1.999 

.101 

Colleague 
Pressure  to 
be  Safe 

Encouraged  to 
Innovate  .093 

.037 

2.514 

.012 

Mean  =  2.219 
SD    =  1.018 

R  Square  =  .016 

n  =  389 

Dependent 
Variable 

Causal 

Variable  B 

SE  B 

T 

Prob. 

Constant  1.502 

.473 

Gains  in 
Lockout 
Knowledge 
Retained 

Employee 

Accoutable  for 

Not  Locking  out  .463 

.206 

2.254 

.025 

Mean  =  2.482 
SD    =  3.691 

R  Square  =  .018 

N  =  284 
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Abstract 

The  effects  of  animated  presentations  on  intentional 
and  incidental  learning  was  studied.  Also  studied 
was  the  degree  to  which  computer  practice  activities 
contained  intrinsically  motivating  characteristics  as 
measured  by  continuing  motivation.  A  total  of  70 
fourth  graders  participated  in  an  introductory  lesson 
on  Newton's  Laws  of  Motion.  Two  levels  of  Visual 
Presentation  (Static  Graphic,  Animated  Graphic) 
were  crossed  with  two  bvels  of  Practice  Order 
(Qu  es  tio  ns/Simu  la tio  n,  Simulatio  n/Ques  tio  ns), 
Within-subjects  factors  consisted  of  Learning  Intent 
(Intentional,  Incidental),  Visual  Testing  Format  (All 
Verbal,  Verbal/Visual),  and  Test  Inten/al  (Immedi- 
ate, Delayed).  Results  showed  that  Animated 
Graphics  successfully  promoted  incidental  learning. 
In  addition,  students  oven^helmingly  chose  to  return 
to  the  structured  simulation  practice  when  placed  in 
a  free-choice  situation. 


Recent  advances  in  authoring  computer- 
based  instruction  (CBI)  have  provided  designers 
with  relatively  easy  means  of  adding  diverse  and 
elaborative  features  to  instructional  software. 
Features  which  once  demanded  advanced  pro- 
gramming skills  and  liberal  development  time- 
lines can  now  be  almost  casually  incorporated. 
Animation  has  long  been  a  popular  feature  in 
CBI,  and  its  use,  along  with  other  types  of 
graphics,  is  often  encouraged  (Bork,  1981; 
Caldwell,  1980).  Unfortunately,  animation  is 
frequently  added  to  CBI  without  serious  concern 
for  its  true  instructional  purpose  or  impact. 

Instructional  research  has  focused  on  two 
fundamental  applications  of  animation  in  CBI: 
1)  animation  as  a  presentation  strategy;  and  2) 
practice  activities  involving  interactive  ani- 
mated graphics.  Recent  reports  have  offered 
encouraging  results  for  both  applications 
(Rieber,  in  press-b;  Rieber,  Boyce,  &  Assad,  in 
press).  General  theoretical  suppor.  of  animated 
visuals  is  provided  by  the  dual-coding  hypothe- 
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sis  (Paivio,  1986)  which  suggests  that  highly 
visual,  concrete  information  is  stored  both  ver- 
bally and  visually.  Retention  can  be  enhanccxi 
by  external  pictures  if  they  promote  the  activa- 
tion of  dual-coding.  When  information  is  dual- 
coded,  the  probability  of  retrieval  is  increased 
since  if  one  memory  trace  is  lost,  another  is  still 
available  (Kobayashi,  1986).  Efficacy  of  ani- 
mated presentations  appears  particularly  de- 
pendent on  task  requirements,  cognitive  load, 
and  attention-gaining  devices.  In  addition  to 
gains  on  intentional  learning,  Rieber  (in  press-b) 
reported  initial  evidence  of  animation's  role  in 
contributing  to  incidental  learning.  Elementary 
school  students  demonstrated  modest  success  in 
attending  to  incidental  information  contained  in 
a  single  animated  display.  A  need  exists  to 
validate  this  preliminary  finding  and  investigate 
its  implications  further. 

The  study  of  incidental  learning  is  well 
established  (Ackerman,  1985;  Goldberg,  1974; 
Klauer,  1984;  Lane,  1980;  Sagaria  &  Di  Vesta, 
1978).  Past  rese^ch  has  primarily  focused  on 
interactions  between  text  processing  activities, 
such  as  pre-  ar.d  post-questions,  and  intentional 
and  incidental  learning.  The  search  for  true 
**mathemagenic"  activities,  or  activities  which 
give  **birth"  to  learning  in  general  (Rothkopf, 
1966),  has  led  researchers  to  understand  that 
most  instructional  activities  incur  tradeoffs  be- 
tween intentional  and  incidental  learning  — 
contiibutions  to  one  generally  impede  the  other. 
Most  of  this  research  has  concentrated  on  learn- 
ing from  prose,  although  several  imagery  stud- 
ies have  b^n  reported  which  indicate  that  recall 
of  incidental  information  is  more  robust  for 
pictures  than  for  words  (Ghatala  &  Levin, 
1981).  Traditional  CBI  design,  based  on  sys- 
tems approaches  (see  Reigeluth,  1983  for  a 
review),  has  only  been  concerned  with  inten- 
tional learning,  that  is,  the  extent  to  >^'hich  prede- 
termined learning  objectives  have  been  mas- 
tered. Unfortunately,  this  approach  can  also 
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lead  to  instructional  designs  which  are  unneces- 
sarily self-limiting  and  restricting  and  which  are 
apt  to  assign  a  passive  role  to  the  learner.  The 
ability  of  the  learner  to  elaborate  upon  instruc- 
tion as  well  as  induce  rules  and  principles  not 
intentionally  taught  is  usually  overlooked  or 
ignored.  The  first  objective  of  this  study  was  to 
investigate  whether  students  would  be  able  at- 
tend to  incidental  information  contained  in  ani- 
mated visuals  and  induce  a  resulting  physical 
science  principle  from  this  information  without 
sacrificing  intentional  learning. 

Previous  research  has  demonstrated  that 
animated  presentations  interact  with  the  practice 
activities  provided  in  CBI.  Traditional  CBI  has 
used  questioning  strategies  as  successful  prac- 
tice activities  (eg.  Hannafin,  Phillips,  &  Tripp, 
1986).  Rieber  (in  press-b)  and  Rieber,  Boyce, 
and  Assad  (in  press)  compared  the  effects  of 
questioning  strategies  to  structured  simulations 
as  practice  activities  in  science  instruction.  The 
structured  simulations  consisted  of  animated 
graphics  which  obeyed  the  laws  of  motion  and 
were  under  student  control.  Brown  (1983)  has 
termed  these  applications  of  animated  graphics 
as  "interactive  dynamicc".  The  graphics  change 
continuously  over  time  depending  on  student 
input,  thus  acting  as  a  form  of  instantaneous 
feedback  (Reed,  1985).  Adult  performance 
compared  equally  with  both  forms  of  practice, 
while  elementary  school  children  performed 
best  given  the  experiences  with  the  interactive 
dynamics.  It  is  speculated  that  the  success  of  the 
interactive  dynamics  with  children  was  due  in 
part  to  the  apparent  intrinsic  motivational  appeal 
of  the  simulation  —  students  were  encouraged 
to  fantasize  about  being  the  pilot  of  a  starship  in 
outerspace.  Intrinsically  motivating  instruction 
relies  on  student-driven  strategies,  rather  than 
external  lesson  reinforcement  (Lepper,  1985; 
Malone,  1981).  Theories  concerning  intrinsi- 
cally motivating  instruction  are  based  on  moti- 
vation research  which  suggests  that  students 
who  see  themselves  in  control  of  their  success 
are  destined  to  stay-on  task  longer  in  more 
meaningful  ways  and  are  more  likely  to  return  to 
the  instructional  task  given  the  freedom  to 
choose  (Malone,  1981;  Rotter,  1954;  Weiner, 
1979).  The  second  objective  of  this  study  was  to 
validate  whether  or  not  these  structured  com- 
puter simulations  hold  intrinsically  motivating 
characteristics  for  children.  Measures  of  con- 
tinuing motivation,  such  as  choosing  to  return  to 
an  instructional  task  in  a  free-choice  situation. 


ERIC 


have  been  used  successfully  to  estimate  the 
construct  of  intrinsic  motivation  (eg.  Kinzie  & 
Sullivan,  1989).  Continuing  motivation  can  be 
defined  as  an  individual's  willingness  to  return 
to  an  activity  once  external  pressure  to  do  so  has 
ceased  (Maehr,  1976). 

There  were  two  purposes  to  this  study:  1) 
to  investigate  the  effects  of  computer  animatea 
presentations  on  intentional  and  incidental 
learning;  2)  to  investigate  the  locus  of  motiva- 
tion to  participate  in  computer-based  practice 
activities.  It  was  hypothesized  that  students 
given  lessons  containing  animated  presenta- 
tions would  significantly  outperform  other  stu- 
dents on  measures  of  incidental  learning.  It  was 
also  hypothesized  that  students  would  choose  to 
return  to  practice  activities  involving  interactive 
dynamics  more  often  than  other  computer  prac- 
tice activities  or  even  other  high-interest,  non- 
lesson  activities,  such  as  paper  puzzles. 

Methods 

Subjects 

The  subjects  consisted  of  70  fourth  grade 
students  from  an  urban  public  elementary 
school  Participation  was  voluntary  with  selec- 
tion based  on  parent  consent. 

CBI  Lesson  Content 

The  CBI  lesson,  fi:  st  developed  for  previ- 
ous studies  (Rieber,  in  press-a,  in  press-b; 
Rieber  &  Hannafin,  1988)  but  modified  for  this 
study,  described  and  explained  Newton's  laws 
of  motion.  The  lesson  was  divided  into  two 
parts.  Part  one  introduced  students  to  Isaac 
Newton  and  his  first  law  of  motion.  Part  two 
provided  applications  of  Newton's  first  law  and 
an  initial  application  of  Newton's  second  law 
where  mass  is  held  constant  and  forces  vary.  All 
information  was  presented  at  an  introductory 
level  in  the  context  of  one-dimensional  motion. 

Lesson  Versions 

Two  levels  of  Visual  Presentation  (Static 
Graphic,  Animated  Graphic)  were  crossed  with 
two  levels  of  Practice  Order  (Questions/Simula- 
tion, Simulation/Questions). 

Visual  Presentation.  In  each  Visual 
Presentation  condition  the  lesson  narrative  was 
supplemented  with  either  static  or  animated 
graphics.  In  the  Static  Graphics  condition, 
forces  were  represented  by  a  foot  giving  an 
object  a  kick,  such  as  a  ball  or  a  concrete  block. 
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FIGURE  1.  Example  of  a 
computer  screen  which 
used  animation  to  present 
intentional  and  incidenMl 
information.  The* con- 
crete block"  and  the  "ball" 
were  both  kicked  once. 
The  animation  depicted  the 
ball's  speed  as  dramati- 
cally greater  than  that  of 
the  block. 


Even  the  smallest  forc^,  like  one  kick,  would  make  a 
big  object  (like  a  concrete  block)  move  forever. . . 

...or  until  another  EQUAL  kick  in  the  OPPOSITE 
direction  makes  the  block  stop. .  . 

. . .  just  like  the  soccer  ball! 

Press  <SPACE  BAR>  after  studying 


Motion  and  trajectory  attributes  were  repre- 
sented by  arrows  and  path  lines.  The  Animated 
Graphics  condition  also  used  the  b.'dl,  block  and 
foot  symbols,  however  the  object  was  then  ani- 
mated to  show  the  motion  and  trajectory  result- 
ing from  the  force  applied  to  it.  Both  types  of 
visuals  elaborated  the  textual  information. 
However,  another  application  of  Newton's  sec- 
ond law  was  incidentally  demonstrated  in  the 
Animated  Graphics  condition.  This  law  predicts 
that  when  two  objects  of  different  masses  are 
both  acted  on  by  the  same  iize  foice,  the  accel- 
eration experienced  by  the  object  with  larger 
mass  is  less.  The  computer  animated  a  large 
solid  box  moving  very  slowly  across  the  com- 
puter screen  after  being  kicked  with  the  same 
size  force  which  had  just  been  applied  to  a  small 
animated  ball,  similar  to  what  is  illustrated  in 
Figure  1.  No  attempt  was  made  to  teach  or 
explain  why  the  box  moved  more  slowly  than 
the  ball. 

Practice  Order.  Each  level  of  Practice 
Order  consisted  of  giving  all  students  two  types 
of  practice  activities,  but  in  different  order.  The 
first  practice  activity  consisted  of  multiple- 
choice  questions.  The  questions  covered  the 
relevant  material  in  the  respective  lesson  part, 
feedback  was  provided  to  the  students  in  the 


form  of  knowledge  of  correct  results.  The  sec- 
ond practice  activity  involved  the  student  in  a 
structured  simulation.  Students  were  given  in- 
creasing levels  of  control  over  a  simulated,  free- 
floating  object  called  a  "starship"  in  the  lesion. 
This  structured  simulation,  an  application  of  the 
dynaturde  microworld  developed  by  diSessa 
(1982)  and  White  (1984),  had  been  shown  to 
positively  affect  lesson  performance  in  prior 
research  with  chiJ.dren  of  this  age  (Rieber,  in 
press-b).  Students  participated  in  the  practice 
activities  in  the  necpective  order  (Questions/ 
Simulation  or  Simulation/Questions)  after  each 
of  tlie  two  lesson  parts.  It  was  important  that  all 
students  have  equal  opportunities  to  experience 
both  forms  of  practice  so  that  interpretation  of 
the  free-choice  data  would  be  valid.  However,  it 
was  anticipated  that  the  first  activity  might  ori- 
ent students  to  the  second  activity,  thus  resulting 
in  different  success  patterns  for  each  activity. 
This  could  affect  the  motivating  appeal  for  each 
activity.  Therefore,  Practice  Order  was  added  to 
the  experimental  design  to  prevent  this  from 
confounding  the  study.  It  was  not  the  intention 
of  this  experiment  to  study  the  performance 
effects  of  each  practice  activity,  as  this  has  been 
adequately  demonstrated  elsewhere  (e.g. 
Rieber,  in  press-b). 
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Dependent  Measures 

PosttesL  The  24-item  posttest  measured 
two  types  of  rule  learning:  intentional  and 
incidental  Intentional  learning  is  defined  as 
learning  those  objectives  which  were  directly 
taught  in  the  lesson  through  the  combination  of 
veibal  (text)  and  visual  means.  Incidental  learn- 
ing is  defined  as  learning  those  objectives  which 
were  not  directly  taught,  but  only  implied 
through  contextual  cues  provided  in  several  of 
the  animated  visual  displays.  The  posttest  was 
administered  twice:  first,  in  two  parts  immedi- 
ately upon  completion  of  the  relevant  lesson 
parts,  and  then  again  approximately  two  days 
later  in  whole  as  a  measure  of  learning  decay/ 
durability.  The  use  of  visuals  in  the  testing  was 
also  controlled.  Half  of  the  intentional  and 
incidental  questions  were  presented  in  an  all- 
verbal  form  and  half  contained  accompanying 
visuals.  Two  questions  were  added  to  the  final 
administration  of  the  posttest  which  tested  a 
simple  application  of  Newton's  Law  of  Gravita- 
tion. ITiis  objective  was  not  taught  in  any  form 
during  the  lesson.  However,  it  was  anticipated 
that  theanimated  visuals  mightpromote  themis- 
conception  that  two  objects  of  different  masses 
would  fall  at  different  speeds  when  dropped  to 
^he  earth.  These  two  questions  provided  a  pre- 
liminary measurement  of  whether  such  a  mis- 
conception was  actually  formed  by  students. 

KR-20  reliability  of  all  50  questions  was 
.93.  Respective  reliability  coefficients  and 
number  of  questions  for  each  of  the  subscales 
were  as  follows:  .87,  24  (Immediate);  .87,  24 
(Delayed);  .91,  28  (Intentional);  .95,  20  (Inci- 
dental); .91, 26  (Veibal);  .85, 22  (Visual);  .64, 2 
(Law  of  Gravitation). 

Response  Latency,  The  time  required  by 
students  to  process  each  of  the  posttest  questions 
was  collected.  Processing  time  is  measured  as 
the  time  (in  seconds)  taken  by  students  to  press 
^he  appropriate  key  once  prompted  to  do  so. 
This  variable  provides  an  important  indirect 
measure  of  the  organization  strength  of  initial 
encoding  as  evidenced  by  retrieval  time. 

Continuing  Motivation.  Immediately 
after  completing  the  second  administration  of 
the  posttest  on  the  fourth  day,  students  were 
given  three  choices:  1)  answer  more  practice 
questions  just  for  fun;  2)  spend  more  time  in  the 
structured  simulation;  or  3)  complete  a  word- 
find  puzzle  on  another  topic.  Students  were  told 
that  they  could  switch  between  activities  if  they 
wished  to  do  so.  Continuing  motivation  of  the 
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practice  activities  was  measuied  by  fu*st  choice 
frequencies  of  these  two  options  as  compared  to 
the  first  choice  frequency  of  the  puzzle  activity. 

Procedures 

All  instruction  and  testing  was  admini- 
stered by  computer  over  the  span  of  four,  30 
minute  computer  sessions.  The  first  session 
acted  as  a  general  orientation  to  familiarize 
students  to  the  materials  and  nature  of  the  proj- 
ect. The  second  session  presented  lesson  part 
one,  followed  by  practice  and  then  12  posttest 
questions  relevant  to  that  part.  The  third  session 
presented  lesson  part  two,  followed  by  practice 
and  then  12  posttest  questions  relevant  to  part 
two.  The  fourth  session  involved  a  re-admini- 
stration of  the  original  24  posttest  questions  and 
2  additional  questions  which  measured 
Newton's  law  of  gravitation.  As  already  indi- 
cated, these  two  questions  were  added  as  a 
preliminary  measure  of  whether  the  animated 
graphics  promoted  niisconceptions.  After  the 
posttest,  students  were  given  the  choice  to  woric 
on  either  of  the  two  computer-based  practice 
activities  (questions  or  simulation)  or  an  irrele- 
vant, though  highly  motivating  activity  consist- 
ing of  a  word-find  puzzle.  Each  sessiou  lasted 
approximately  30  minutes  amounting  to  about 
120  minutes  of  learner  time  over  the  course  of 
about  10  days.  Students  were  randomly  as- 
signed to  one  of  the  treatment  groups  as  they 
arrived  at  the  computer  lab  for  the  first  session. 
Once  instructed  to  begin  at  the  start  of  each 
session,  students  completed  the  lesson  individu- 
ally under  controlled  conditions. 

Design 

In  this  study  a  2  X  2  X  (2  X  2  X  2)  factorial 
design  was  used.  Two  between-subjects  factors 
levels  (Visual  Presentation,  Practice  Order) 
were  crossed  with  three  within-subjects  factors 
(Learning  Intent,  Visual  Testing  Format,  Test 
Time).  The  between-subjects  factors  consisted 
of  two  levels  of  Visual  Presentation  (Static 
Graphic,  Animated  Graphic)  and  two  levels  of 
Practice  Order  (Questions/Simulation,  Simula- 
tion/ Questions).  The  within-subjects  factors 
consisted  of  two  levels  of  Learning  Intent  (In- 
tentional, Incidental),  two  levels  of  Visual  Test- 
ing Format  (All  Verbal,  VerbaWisual),  and  two 
levels  of  Test  Time  (Immediate,  Delayed).  A 
mixed  factor  analysis  of  variance  (ANOVA) 
was  used  to  analyze  the  quantitative  data.  The 
within-subjects  factors  also  served  as  dependent 
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TABLE  1 .  Mean  Percent  Scores  and  Standard  Deviations  for  the  Postiest 


Im;nc(<iaic  Postlcst 


Intentional  Incidental 


Visual 
Presentation 

Practice 
Order 

Verbal 

Verbal/ 
Visual 

Verbal 

Verbal/ 
Visual 

Animated 
Graphic 

Questions/ 
Simulation 

Mean 

SD 

n 

73.5 
(30.7) 
17 

69.9 
(27.3) 
17 

66.4 
(29.4) 
17 

76.5 
(28.3) 
17 

Simulation/ 
Questions 

Mean 

SD 

n 

75.5 
(24.4) 
17 

69.1 
(26.2) 
17 

59.7 
(32.0) 
17 

76.5 
(36.8) 
17 

Static  Graphic 

Questions/ 
Simulation 

Mean 

SD 

n 

61.8 
(21.9) 
17 

53.7 
(27.2) 
17 

15.1 
(13.8) 
17 

29.4 
(28.6) 
17 

Simulation/ 
Questions 

Mean 
SD 

1^ 

66.7 
(35.1) 
19 

52.6 
(31.6) 
19 

15.0 
(25.0) 
19 

22.8 
(31.5) 
19 

Delayed  Posttcsl 

Intcnlional 

Incidental 

Visual 
Presentation 

Practice 
Order 

Verbal 

Verbal/ 
Visual 

Verbal 

Verbal/ 
Visual 

Animated 
Graphic 

Questions/ 
Simulation 

Mean 

SD 

n 

74.5 
(28.3) 
17 

81.6 
(18.8) 
17 

69.7 
(27.6) 
17 

70.6 
(37.0) 
17 

Simulation/ 
Questions 

Mean 

SD 

n 

68.6 
(25.6) 
17 

79.4 
(18.2) 
17 

65.5 
(27.7) 
17 

66.7 
(37.3) 
17 

Static  Graphic 

Questions/ 
Simulation 

Mean 

SD 

n 

52.9 
(21.4) 
17 

59.6 
(28.5) 
17 

16.0 
(23.6) 
17 

J  7.6 
(33.6) 
17 

Simulation/ 
Questions 

Mean 

SD 

n 

54.4 
(37.6) 
19 

59.2 
(32.8) 
19 

10.5 
(24.2) 
19 

15.8 
(28.0) 
19 

Notes:  SD-standard  deviation;  n-numbcr  of  subjects 
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RGURE  2.  Interaction  between 
Visual  Presentation  and  Learning 
Intent. 
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measures.  A  Chi  Square  analysis  was  used  to 
test  students'  first  choice  preferences  of  each 
post-lesson  activity  as  a  measure  of  continuing 
motivation  of  the  practice  questions  and  the 
simulation. 

In  addition,  separate  analyses  were  con- 
ducted on  the  questions  measuring  misconcep- 
tions of  Newton's  law  of  gravitation  and  the 
response  latency  data  from  the  posttest. 

Results  and  Discussion 

Posttest 

Percent  means  and  standard  deviations  of 
student  posttest  scores  are  contained  in  Table  1. 
An  a  priori  decision  was  made  to  ignore  interac- 
tions involving  three  or  more  factors.  A  signifi- 
cant mai.i  effect  was  found  for  Visual 
Presentation.  F(l,66)=60.17,  p=.001,  MStt^ 
ix>i<=2649.37.  However,  inteipretation  of  this 
effect  must  be  qualified  because  of  its  interac- 
tion with  Learning  Intent,  66)=  18.73, 
p=.0001,  Af5crTor=2249.61.  Figure  2  illustrates 
the  nature  of  this  interaction.  Students  in  the 
Animated  Graphics  group  perfonricd  similarly 
on  both  intentional  (mean=74.2%)  and  inciden- 
tal questions  (mean=67.5%),  whereas  students 
in  ihe  Static  Graphics  group  performed  signifi- 
cantly better  on  intentional  learning  questions 
(mean=57  4%)  than  incidental  questions 
(mean=:16.3%).  As  exf^ted,  these  latter  stu- 
dents performed  at  about  the  guessing  level  on 
incidental  questions.  This  result  supports  the 
first  hypothesis  of  this  study  that  students  re- 


ceiving animated  demonstrations  of  incidental 
information  would  attend,  understand,  and  re- 
trieve this  information  when  tested  although  no 
formal  attempt  was  made  to  teach  or  explain  tlie 
principle.  The  superiori'fy  of  the  Animated 
Graphics  group  over  the  Static  Graphics  group 
on  intentional  learning  is  consistent  with  previ- 
ous research  (Rieber,  in  press-b). 

Among  the  within-subjects  factors,  sig- 
nificant main  effects  were  found  for  Learning 
Intent,  F(l,66)=31.28,  p=.0001,  MStt- 
ror=2249.6l,  and  Visual  Testing  Format, 
F(l,66)=5.39,p=.02,  Af5cnor=306.59.  Not  sur- 
prisingly, students  scored  higher  on  questions 
measuring  Intentional  Learning  (mean=65.6%) 
than  questions  measuring  Incidental  Learning 
(mean=41.1%).  Students  scored  higher  on 
questions  contFiining  either  a  static  or  animated 
visual  (mean=60.1%)  than  questions  containing 
no  visual  (mcan=51.4%).  However,  a  signifi- 
cant interaction  was  found  between  these  two 
factors,  F(!,66)=5.62,  p=.02,  M5cm)f=357.52. 
Students  tended  to  answer  Incidental  Learning 
questions  which  contained  visuals  more  accu- 
rately than  questions  which  contained  no  visu- 
als. Student  scores  on  questions  measuring 
Intentional  Learning  did  not  vary  significantly 
depending  on  the  use  of  visuals  in  the  questions. 
This  implies  that  in  this  study,  the  use  of  visuals 
in  ihe  test  construction  did  not  measure  differen- 
tiated amounts  of  intentional  rule  learning. 
However,  test  questions  which  included  visuals 
did  vary  in  their  measurement  of  incidental  rule 
learning. 
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Response  Latenc/ 

A  main  effect  was  found  for  Test  Time, 
F(l,66)=:46.13,  p^.OOOl,  Af5crror=35.99.  Stu- 
dents took  significantly  less  time  to  answer  the 
posttest  questions  during  the  second  administra- 
tion of  the  posttest.  Retrieval  time  would  be 
expected  to  decrease  during  retesting,  especially 
when  the  second  administration  occurred  within 
48  hours,  as  was  the  case  in  this  study.  However, 
an  ordinal  interaction  was  found  between  Test 
Time  and  Visual  Presentation,  F(l,66)=:5.48, 
/?=:.02,  Af5crTor=35.99.  Students  in  the  Statics 
Graphics  group  varied  less  in  response  time 
from  the  first  and  second  administration  of  the 
posttest  than  students  in  the  Animated  Graphics 
group.  This  interaction  is  probably  an  artifact  of 
the  rate  of  guessing  on  incidental  questions  by 
the  Static  Graphics  group. 

A  main  effect  was  found  for  Learning 
Intent,  F(l,66)=80.89,  p=.0001,  Af5crTor=30.80. 
Students  took  significantly  less  time  to  answer 
questions  measuring  Intentional  Learning  than 
questions  measuring  Incidental  Learning. 
However,  an  ordinal  interaction  was  also  de- 
tected between  Learning  Intent  and  Visual 
Presentation,  F(l,66)=8.77,  p=.004,  MStv 
ror<=30.80.  Students  in  the  Animated  Graphics 
group  took  significantly  longer  to  answer  ques- 
tions measuring  Incidental  Learning  than  stu- 
dents in  the  Static  Graphics  group.  This  was  ex- 
pected. Students  in  both  Visual  Presentation 
groups  were  required  to  overtly  process  inciden- 
tal information  for  the  first  time  during  the 
posttest.  Since  students  in  the  Static  Graphics 
group  had  to  revert  to  either  guessing  or  personal 
experience,  their  latency  rates  for  these  ques- 
tions would  be  expected  to  be  low^.r  than  for 
students  in  the  Animated  Graphics  gicup.  These 
latter  students  were  asked  to  answer  questions 
pertaining  to  information  which  was  only  inci- 
dentally represented  in  the  animated  visuals.  If 
students  encoded  the  incidental  information  sig- 
nificant time  would  be  necessary  to  retrieve  this 
information  and  formulate  a  rule  which  could  be 
used  to  answer  the  question.  There  was  no 
difference  in  response  time  between  Animated 
Graphics  and  Static  Graphics  groups  for  Inten- 
tional Learning. 

A  main  effect  was  found  for  Visual  Test- 
ing Format,  F(l,66)=:126.97,  p=:.0001,  M5cr. 
rof=3 1.15.  Overall,  students  took  much  less  time 
to  respond  to  questions  which  contained  either 
static  or  animated  visuals  than  questions  which 
contained  no  visual  supplements.  This  too  was 


expected.  All  of  the  information  and  testing  was 
highly  visual  in  nature.  Questions  which  did  not 
provide  external  visuals  would  require  students 
to  form  their  own  internal  or  external  (eg. 
doodles  on  paper)  images  in  order  to  solve  the 
problems.  When  visuals  were  provided  with  the 
questions,  students  could  by-pass  this  time-con- 
suming cognitive  step,  assuming  that  the  visual 
was  congruent  with  each  student's  mental 
schema.  It  is  insightful  to  compare  the  latency 
and  performance  results  for  this  factor.  Even 
though  the  addition  of  external  visuals  to  ques- 
tions which  measured  intentional  learning  did 
not  contribute  to  performance  differences,  the 
decrease  in  latency  suggests  that  these  visuals 
nonetheless  aided  the  cognitive  processing  nec- 
essary for  accurate  responses. 

Continuing  Motivation 

Respective  first  choice  frequencies  for 
each  of  the  three  post-lesson  activities  were  as 
follows  (the  first  number  indicates  the  raw  fre- 
quency and  the  second  number  indicates  per- 
centage of  total  first  choices):  Questions,  3, 
4.3%;  Simulation,  42,  60.0%;  Puzzle,  25, 
35.7%.  For  the  purposes  of  the  C.:^  Square 
analysis,  answering  more  practice  questions 
acted  as  a  non-choice  and  was  removed  from  the 
analysis.  Overall,  students  first  chose  to  partici- 
pate in  the  structured  simulation  more  often  than 
either  the  puzzle  activity  or  the  practice  ques- 
tions, Chi  Square=:3.88,p<.05,4r=l.  This  result 
supports  the  second  hypothesis  of  the  study 
which  contended  that  the  practice  provided  by 
the  structured  "starship"  practice  held  intrinsi- 
cally motivating  appeal  for  the  students.  It  is 
contended  that  this  appeal  would  lead  students 
to  choose  to  stay  on  task  longer  in  meaningful 
ways.  This  result  further  clarifies  performance 
effects  of  this  practice  activity  as  reported  by 
Rieber  (in  press-b). 

Student  Conceptions  of  Gravitation 

A  significant  main  effect  was  found  for 
Visual  Presentation,  F(l,66)=:18.24,  /7=.0001, 
Af5cnx>r=:1044.19.  Students  in  the  Animated 
Graphics  group  (mean=5.9%)  performed  sig- 
nificantly worse  on  the  two  questions  measuring 
a  simple  application  of  Newton's  law  of  gravita- 
tion than  students  in  the  Static  Graphics  group. 
This  suggests  that  students  in  the  Animated 
Graphics  group  were  more  prone  than  other 
students  to  the  misconception  that  Newton's 
second  law  predicts  the  free  fall  characteristics 
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of  objects  of  different  masses  on  earth.  This 
result  provides  an  important  counterbalance  to 
the  apparent  ability  of  students  to  attend  to 
animated  visuals  which  incidentally  demon- 
strates a  scientific  principle.  While  students 
may  be  appropriately  attending  to  the  details, 
they  may  he  drawing  a  whole  series  of  conclu- 
sions —  some  of  which  are  appropriate,  and 
some  of  which  are  not.  This  phenomena  is 
consistent  with  previous  research  on  other  mis- 
conceptions in  science  where  people  formulate 
conceptions  of  physical  phenomena  based  on 
daily  observable  events  (see  Eylon  &  Linn, 
1988,  for  a  review).  Without  direct  guidance, 
such  as  that  provided  by  instruction,  students 
may  formulate  innumerous  and  potentially  erro- 
neous "theories"  about  the  physical  worid. 

General  Discussion 

There  were  two  purposes  to  this  study. 
First,  the  ability  of  students  to  fust  attend  to 
incidental  information  contained  in  an  animated 
visual,  and  then  extract  and  induce  science  prin- 
ciples was  investigated.  The  Animated  Graph- 
ics group  was  given  demonstrations  of  an 
extension  of  Newton's  second  law  where  the 
masses  of  two  objects  varied,  but  the  forces 
acting  on  them  remained  constant.  However, 
the  fact  that  these  demonstrations  represented  a 
distinct  extension  of  this  principle  was  never 
brought  to  their  attention.  In  fact,  the  demon- 
strations were  presented  in  the  context  of  ex- 
plaining other  lesson  content.  It  is  believed  that 
attention  to  intentional  learning  alone  placed 
significant  cogniti>^e  loads  on  students.  Yet, 
they  were  able  to  gain  understanding  of  the 
incidental  information,  without  apparently  sac- 
rificing any  intentional  learning.  The  results 
reported  here  demonstrate  the  rather  startling 
ability  of  fourth  grade  students  to  attend  to, 
formulate,  and  apply  even  a  sophisticated  sci- 
ence principle  introduced  in  an  incidental  fash- 
ion. 

However,  this  result  is  tempered  by  the 
alarming  number  of  students  who  continued  to 
apply  this  incidental  information  to  other,  inap- 
propriate contexts.  Students  in  the  Animated 
Graphics  condition  were  much  more  likely  to 
hold  misconceptions  about  the  effects  of  gravity 
on  objects  of  different  masses.  This  is  not  really 
surprising.  The  animated  demonstrations  of 
Newton's  second  law  occurred  in  a  simulated 
gravity-free/frictionless  environment.  The 


concept  and  effect  of  gravity  was  purposely 
removed  from  the  discussion.  Without  addi- 
tional guidance,  it  is  logical  and  intuitive  to 
apply  Newton's  second  law  to  contexts  where 
objects  of  different  masses  are  freely  failing  on 
earth  (such  an  intuitive  assumption  led  Galileo 
to  demonstrate  otherwise  at  the  leaning  Tower 
of  Pisa). 

The  second  purpose  of  this  study  was  to 
estimate  the  construct  of  intrinsic  motivation  as 
it  related  to  several  practice  activities  shown 
effective  in  CBI  in  previous  research  —  ques- 
tioning strategies  and  structured  simulations. 
Understanding  whether  certain  activities  have 
intrinsic  motivating  appeal  is  important  when 
designing  CBI.  If  activities  contain  such  appeal 
for  students,  they  will  be  more  likely  to  maintain 
interest  and  stay  on-task  for  longer  periods  of 
time.  This  is  because  the  focus  of  the  reinforce- 
ment is  internal  —  that  is,  students  motivate 
themselves.  This  is  contrast  to  traditional  CBI, 
largely  based  on  behavioral  designs,  where 
learning  is  believed  to  be  the  result  of  constant 
and  appropriate  external  reinforcement.  It  can 
be  argued  that  internal  reinforcers  account  for 
most  infonnal  daily  learning  such  as  that  expe- 
rienced through  hobbies  and  sports.  Designers 
of  CBI  practice  activities  have  many  strategies 
from  which  to  choose.  It  is  argued  here  that 
while  several  activities  may  promote  similar 
performance  levels,  activities  which  are  intrinsi- 
cally motivating  carry  other  significant  advan- 
tages such  as  personal  satisfaction,  challenge, 
and  relevance  and  promotion  of  a  positive  per- 
spective on  lifelong  learning  (cf  Keller  & 
Suzuki,  1988).  Previous  i^search  has  used  stu- 
dent free  choice  patterns  as  a  measure  of  con- 
tinuing motivation  (Kinzie  &  Sullivan,  1989). 
m  this  study,  students  overwhelmingly  chose  to 
return  to  the  "starship"  practice  more  often  than 
questioning  activities  or  even  traditionally  en- 
teitaining  activities  such  as  word-find  puzzles  to 
support  tfie  contention  that  the  structured  simu- 
lation carries  intrinsically  motivating  appeal  for 
students  of  this  age.  It  is  further  contended  that 
the  activities,  and  not  the  computer,  possesses 
these  characteristics  based  on  the  fact  that  of  the 
other  two  choices,  the  paper  and  pencil  activity 
was  overwhelmingly  chosen  over  the  remaining 
computer  activity  (questions). 

This  study  has  also  provided  preliminary 
findings  concerning  the  role  of  visuals  in  com- 
puter-based testing.  Testing  learning  outcomes 
which  are  highly  visual  in  nature  poses  problems 
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for  researchers.  A  visual  in  a  test  question  may 
actually  act  as  an  unintentional  learning  aid 
which  could  confound  results.  However,  the 
validity  of  testing  visual  learning  outcomes  with 
solely  verbal  means  can  easily  be  questioned. 
Insights  into  this  dilemma  were  hoped  to  be 
gained  by  controlling  for  visuals  in  the  testing. 
In  this  study,  the  validity  of  questions  measuring 
intentional  learning  was  not  related  to  the  pres- 
ence  of  visuals.  Questions  which  did  not  contain 
visuals  appeared  to  state  the  problem  in  terms 
which  enabled  students  to  form  their  own  appro- 
priate internal  images.  This  was  not  true  when 
measuring  incidental  learning.  Students  appar- 
ently had  difficulty  forming  internal  images  on 
their  own  as  they  attemi/ted  to  solve  the  problem. 
More  work  is  needed  to  understand  the  role  of 
visuals  in  testing  (other  examples  include  Hale, 
Oakey,  Shaw.  &  Bums,  1985;  and  Taylor, 
Canelos,  Belland,  Mayton,  &  Dwyer,  1989), 

Lastly,  students  in  this  study  did  not  expe- 
rience significant  learning  decay  over  the  two 
day  test-retest  interval.  Previous  studies  have 
indicated  that  picture  effects  are  durable  over 
long  periods  of  time  (Anglin,  1986;  Peng  & 
Levin,  1979).  The  results  reported  here  support 
this  contention. 

In  conclusion,  this  study  has  shown  one 
example  of  the  powerful  ability  of  students  tu 
learn  in  incidental  ways.  Designers  of  CBI  need 
to  harness  this  ability  by  promoting  it  whenever 
possible  while  trying  to  avoid  inappropriate 
student  elaborations.  One  application  of  this 
research  to  the  development  of  CBI  may  be  to 
use  incidental  information  as  an  onenting  activ- 
ity for  subsequent  intentional  instruction-  More 
research  is  needed  to  understand  how  to  best 
incorporate  incidental  learning  benefits  into  the 
design  of  CBI.  Additionally,  this  study  has 
provided  strong  evidence  to  suggest  that  certain 
computer  practice  activities  contain  intrinsi- 
cally motivating  appeal  for  elementary  school 
students.  Future  research  should  be  directed  at 
replicating  these  results  in  other  contexts  and 
content  areas. 
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ABSTRACT 

The  effects  of  overt,    wOvert,   and  summary  practice 
cot^ditions  on  four    levels  of    learning  were  examined.     On^-  hundred 
twenty  two  middle  class  sixth  grade  students  were  randomly 
assigned  to  practice  condition.     Passages  regarding  energy 
sources  were  read  by  the  students  and  oral    reviews  were  directed 
according  to  treatment.     The  summary  group  was  given  a  review  of 
relevant  content,    while  covert  and  overt   groups   responded  to 
adjunct  questions.      A  thirty  two   item  selected  response  test 
ccrtained  sixteen   i  teirs   froir  the  practice  booklet  and  sixteen  new 
:tem=    not   practiced  by  any  treatment.      Data  were  examined  on  four 
dependant  measures   (repeated  fact,    new  fact,   repeated  inference 
new  inference).     The  results  show  that  oral   summaries  of  relevant 
information  can  be  as  effective  as   inserted  questions  on 
immediate  and  delayed  test  performance.     This   study  suggests  that 
practice  condition  and  question  type  may  differentially  affect 
learning  and  retention. 


I ntroduct  i  on 

Cognitive  theory  generally  holds  that  recall   of  textual 
information   is   strongly  affected  by  the   level   of  information 
processing   (Craik  &  Lockhart   1972,   Bretzing       Kulhavy  1979). 
Based  on   the  theory  that   recall   of   textual    information   is  a 
function  of   processing  it  vas  predicted  that  as  reading 
comprehension  tasks  demanded  a  more  complete   interaction  of  the 
reader's  existing  schemata,   and  as  the  subjects  were  more 
actively  involved   in  the  learning  process,   higher   levels  of 
recall   would  be  found   (Glover  1982). 

An  underlying  assumption   in  most  practice  condition 
research  is  that  questions  are  beneficial   to  learning.  Andre 
(1979)   found  that  questions   influenced  the  nature  of  the  memory 
representations  formed  during  instruction  and  that  these 
representations  controlled  the  way  in  which  stored  knowledge  was 
used.     Andre  and  Sola  (1976),   and  Andre  and  Womack  (1978) 
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suggested  that  adjunct  paraphrased  questions  guide  semantic 
encoding  of  presented  information  and  thereby   lead  to  greater 
performance. 

Many  studies  have  demonstrated  that  treatments  involving 
questions   result   In  higher    learned  performance   (Anderson  & 
BIddle,    1975).     After  reviewing  the  research   in  this  area,  Dillon 
(1982)    reported  that  adjunct  questions  had  a  consistent^  but 
small,    positive  effect  on  the  retention  o^  Information. 

Two  common  practice  conditions  currently   In  the  research  are 
overt   (verbal   or  written  responses)   and  covert  (Internalized 
responses).     Subjects   In  studies  comparing  the  effects  of  these 
practice  conditions  have  been  typically  presented  written  or 
verbal  questions. 

Many  studies   (Goldbeck  Sc  Campbell,    1962;   Holland,  1960; 
Michael   t>c  Maccoby,    1961)   have  shown  that  both  covert  and  overt 
responses  facilitate  student   learning.     Research  has  indicated 
that  when  verbalization  follows  an   instructional   segment,  both 
question  relevant  and   incidental    subject  content   learning  improve 
(Her shberger ,    196A;   Frase,    1968  &  1970). 

Alternatives   to  the  non-question  control    group  have  been  used 
in  many  studies   (Buggey,    1971;   Ryan,    1973).     Winne  (1979) 
suggested  the  use  of  a  summary  group  as  a  realistic  classroom 
approach  and  non-question  treatment.     Research,    however,  has 
provided   little  information  about  the  effects  of  such  treatments. 

Numerous  studies  have  shown  no  significant  differences 
between  h i gher -cogn i t I ve  and  fact  question  teaching  strategies 
(Andre,   Mueller,   Womack,   Smid,   8^  Turtle,    1980;   Ryan,  197A; 
Bedwell,    1975;   Savage,    1972).     Others  too  have  asserted  that 
higher-level   questions  have   little  defi>onst rated   relationship  to 
student  achievement   (Gall    1970;   Dunkir  &  Biddle,    1974).  Frase 
(1968)    reported  that  broader  questions  do  not  direct  the 
learner's  attention  and  therefore  lead  to  poorer  performance. 

Some  studies,   however,   have  shown  that  application-type 
review  questions  facilitate  a  higher-order   learning  because  they 
require  the   learner   to  apply  acquired   information  to  new 
situations.     Yost,   Avila,   and  Vexler   (1977)  reported  that  as  the 
complexity  of  questions  to  which  students  responded  increases, 
achievement   increases.     Their  study  found  that  higher-order 
questions  generally  facilitated  posttest  performance  by  directing 
attention  to  more  information. 

Differential  effects  for  reading  ability  are  well  supported 
in  research  on  adjunct  questions.     High  readers  generally  score 
higher  on  content  relevant  tests  than  low  readers.  Question 
placement  and  question  type  greatly  affect  results  (Richards  &  Di 
Vista,    1974).     Many  studies  have  found  faci Illative  effecs  of 
question^  across  reading  levels* 

Experimental  studies  investigating  Immediate  and  delayed 
criterion  test  scores  have  revealed  differentia!  effects  for 
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practice  condition  and  question  type*     Overt  and  covert  responses 
have  been  equally  beneficial   when  testing  occurs  immediately 
following   instruction   (Kanner  &  Sulzer,    1956).     However,  time 
delay  in  the  administration  of  the  criterion  measure  has  revealed 
that  overt   responses  are  significantly  more  effective  than  covert 
(Goldbeck  &     Campbe 1 1 , 1962 ) •     Anderson   (1970)   states  that  due  to 
depth  of  processing,   h i ghe r -cogn i t i ve  questions  are  retained 
longer  than   lower-cognitive  questions. 

The  purpose  of   this  study  was  to   investigate  the  effect  of 
overt,    covert,   and  summary  conditions  on  four   levels  of  student 
learning.    A  32  item  posttest  assessed  the  following:  factual 
information  presented  and  practiced   in  the  instruction  (repeated 
fact),    factual    information  presented  but  not  practiced   in  the 
instruction   (new  fact),    information  directly  inferable  and 
practiced   (repeated   inference),    information  directly  inferable 
but   not  practiced   in  the   instruction   (new  inference). 


Method 


Sub  iect s 

The  original    sample  consisted  of   148  sixth  grade  students 
from  two  elementary  schools  that  served  middle  class  suburban 
populations  and  were  of   equal    size.     Twenty  six  subjects  were 
dropped  from  thp  study  for  absence  from  an  experimental  session 
or   for   failure  to  complete  the  attitude  inventory.     The  final 
sample  had  a  total   of   122  subjects. 

Ma  t  e  r  1  a  1  s 

An  instructional   passage  consisting  of  approximately  1100 
words  was  adapted  and  modified  from  the  •'Power  Switch**  unit  of 
the  Energy  Source  Program   (1982).     Passages  described  eight 
potential   energy  sources  of  the  future.     The  text  was  divided 
into  nine  sections,   each  containing  two  or  three  paragraphs. 
Paragraph   length  ranged  from     50  to  65  words.     The  readability 
level    of   the  material   was  5.8  as  determined  by  Brown's  (1965) 
science  text  adaptation  of  the  Dale-Chall   readability  formula. 

Treatment  booklets  were  developed  for  specific  use   in  the 
three  different  practice  conditions.     Summary  group  booklets 
consisted  of  text  only,   while  covert  and  overt  group  booklets 
contained  adjunct  questions.      Inserted  questions  were  located 
Immediately  after  each  passage.     Booklets  were  19  pages  In 
I ength. 

An  Introduction  was   included   in  all  booklets.     All  groups 
were  allowed  practice  on  both  factual   and  inferential  questions. 
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Questions  used  in  this  study  were  subaittsd  to  experts  who 
verif  ed  that   inference  questions  did  indeed  requ  re  students  to 
™i  io   logical   conclusions  by  using  the  information  presented. 
F^ct  qSestfons  w"e  found  to  require  only  simple  verbatim  recall 

QuesUons  and  text  were  submitted  to  a  panel  °f 
Educational  Technology  PhD  candidates.     They  concurred  that  the 
Inference  questions  could  bo  answered  by  using  only  the 

n  o  matlon  in  the  passage.     They  agreed  that  there  was  one  best 
answer  of   the  selected   responses,   and   that  dxstractors  met 
instructional   design  criteria.  tested  with  a 

The  questions   for  this  study  were  first  P^, '  °\ ^^^^^'^         ^  / 
group  tha?  did  not  read  any  of   the  passages.     Questions  used  in 
^Ms  experiment  were  those  which  students  rece  ved  a  chance 
pe^cen^age  of  approximately  twenty-five  percent  correct.  The 
itoms  ind  text  were  then  presented  to  fifty-one  subjects  "ho  . ead 
th^  mateMals  silently  and  answered  all   questions.     From  resu  ts 
o5  ivi  pUot  Vest,   a   pool   of   fact  and   inference  questions,  w.th 
overan   equivalent   difficulty  and  discrimination   indices,  was 

'""'The'criterion   test   consisted  of  32   items.      Sixteer  of  these 
v^e^e  lei   items  and   16  were   repeated  from  the  text.      Fact  and 
V^ference   items  were  represented  equally  and  covered   the  sa.e 
material   but  were  not   parallel  questions. 

Procedures 

Subjects  were  ranked  as  high,   medium,   or    '  ^^^^  ^  „g 

-heir   teachers.      Standardized  scores  were  not  used     or  leveling 
^ubects  because  this   information  was  not  made  available  at  bot.. 
schools       Groups  were  blocked  for   reading   level   and  sex  and 
sublects  were  randomly  assigned  to  groups. 

The  three  treatments  were  administered  by  PhD  cand  dates 
traineS  in  the  experimental   procedures.     Each  experimenter  was 
randomly  assigned'to  a  group  and  presente-i  the  same  practice 

^^^'^Dirinrt^e'^reri^en^^     session  ail   students  were  directed 
tn  the^r  assigned  classrooms.     Experimenters  handed  out  a  pre- 
codiS  pick"   fo  each  subject.      All   students  received   the  same 
general  instructions   including  a  brief   lesson  and  practice  in 
answering  fact  and  inference  questions.     Scripts  for 
experJienters  specified  content,    style  of  delivery  verbatim 
Dhrasings.   and   the  sequence  of  questions  to  be  asked. 
^        Co5e  t  and  overt  groups  were  exposed  to  text,   given  a 
question  to  an.wer.   and  were  called  on  by  the  experimenter  to 
answer  the  question.     Knowledge  of  correct  response  was 
summarized  2y  the  experimenter  before  proceeding  to  the  next 
passage      The  overt  group  was  instructed  to  write  responses  on 
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the   line  at   the  bottorc  of  each  page.     The  covert   group  was  told 
to  mentally  select   the  best  answer.     Questions  were  purposely  not 
labeled  with  a,   b,    c,   or  d.     Therefore,   when  practicing  responses 
or  reporting  to  the  group,    students  were  required  to  repeat  the 
answer   itself  and  not  simply  the   corresponding   letter  or  number. 
The  experimenter  always  responded  to  student  answers  according  to 
the  script.     Both  question  groups  received  corrective  feedback 
but  t)o  supplemental  information. 

The  experimenter   presented  a  scripted   review  to  the  summary 
group.     This   review  covered  the  same  facts  and  concepts  as 
ment  ioned   in  questions  given  to  both  ques  t  i  on  groups .  All 
information  was  presented   in  declarative   form.     Treatment  time 
for  both  question   groups  was  approximately  one  hour  and  15 
minutes.     The  summary  group,   which  did  not   require   teacher  wait 
time  for  selection  of  answers,   calling  on  students,    or  giving 
know*3dge  of   results,    took  only  fifty  minutes.      The  immediate 
criterion   test  was  administered  when  each  group  finished  the 
materials.     The  delayed  test  was   given  exactly  one  week  later. 

Criterion  Test 

The  32   item  selected  response  criterion   test  consisted  cf  16 
repeated   items,    from  question  group  booklets,    and   16  new  items. 
Half   of   the   items  were  fact  questions  and  half   were  inference 
questions.     The   immediate  test  and  delayed   test  were  identical. 

Design  and  Data  Analysis 

A  3   (practice  condition)   X  3   (reading   level)    factorial  design 
was  used.     Data  were  examined  with  four  dependent  measures 
(repeated   fact,    new  fact,    repeated   inference,    new  inference). 
Each  multivariate  analysis  of   variance  was   followed  by  a 
univariate  analysis.      Immediate  and  delayed  tests  were  reviewed 
separate  1 y . 

Resul ts 

Mean  scores  by  pract  i  ce  cond 1 1 1  on,    ques  t  i  on   type,  and 
reading   level   on  the  32   Item   immediate  test  are  presented  In 
Table  1.     Table  2  displays   the  mean  scores  for   the  delayed  test. 
A  summary  table  for  univariate  and  Multivariate  analysis  of 
variance   Is  shown  on  Table  3. 

Results  are  discussed  below  by  immediate   test,    delayed  test, 
and  gain  scores. 


ERLC 


490 

4S4 


Table  1 


Reading 
Level  and 
Condition 
High 


Over  t 


Cover  t 


X 

SD 


SD 


Summary  x 
SD 


Mean  Scores  on  Immediate  Test 


Fact 


Repented 
1  tens 


7.  18 
.  73 

7.06 
1  .  16 

5.92 
1  .26 


New 
I  terns 


4.71 
1 .  40 

4.92 
1.  56 

6.  15 
.90 


I nf erence 


Repeated 

I  tems 


6.  59 
1.28 

5.83 
1.64 

6.  38 

*  '-5 


New 
I tems_ 


4.35 
1.50 

4.  33 
1.23 

5.77 
1.42 


Grand 
Total 


22.82 
3.63 

22.  17 
4.30 

24.23 
3.75 


Med  i  un^ 
Ove r  t 

Cover  t 


SD 

X 

SD 


Summary  x 
SD 


5.  82 
1  .85 

5.73 
1.91 

5.06 
1.52 


4.  88 
1.  76 

4.  33 
1  .95 

5.  94 
1 .  03 


5.  53 
2.  18 

5.  00 
2.  27 

4.  94 

2.  30 


3.  71 
1  .  36 

u.  27 
1.91 

4.  76 
1.79 


19. 9A 
5.10 

19.33 
6.06 

20.  71 
4.  70 


Low 
Overt 

Cov«r  t 


X 

SO 


SD 


Summary  x 
SD 


Total  X 


D.27 
1.56 

4.  40 

2.55 

3.  10 
1.97 


5.64 
1.9A 


Practice  C-9"^^^^o" 

Overt  20.09 
Cov#r*-  18. 79 
Summary  20.23 


3.  36 

2.  38 

3.  50 
1.65 

4.  30 
1  .  25 


4.78 
1-76 


4.  09 
1  .  33 

3.  30 
2.  06 

3.70 
1.64 


5.  19 
2.  09 


Reading  Level 


High 
Hedi  urn 
Low 


23. C-* 
20.0'. 
14.  77 


3.36 
1.43 

2.70 
1.42 

3.  10 
2.  47 


4.  13 
1.79 


IS. 09 
5.  36 

13.90 
5.  49 

14.  20 
6.21 


19.74 


Fact 

Inference 


10.42 
9.  32 
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Immediate  Test 


Pract  i ce  cond  i  1 1  on 

As  shown  in  Table  1,    the  mean  scores  for   the  Immediate 
criterion  test  were  20.09  for  overt,    18.79  for  covert,    and  20.23 
for  summary  groups.     The  multivariate  analysis  of  variance  (Table 
3)   revealed  a  significant  effect   for  practice  condit.jn  (F 
(8,222)=  6.02,  E<.0001). 

Scheffe's  multiple  comparison  test  found  significant 
differences  on  repeated  fact  and  new  fact   items,   but  not  on 
repeated   inference  or  new  inference  items.     On  repeated  fact 
items,    the  mean  scores  of  6.20  for  the  overt  treatment  and  5.81 
for  the  covert  treatment  were  significantly  higher  than  the  mean 
of  4.85  for  the  summary  group,   F   (1,113)=  3.07,    e.<*05.     On  new 
fact  questions  the  situation  was   reversed.     The  mean  of  5.60  for 
the  summary  group  was  significantly  higher   than  the  means  of  A.A4 
for  the  overt   treatment  and  4.29  for   the  covert  treatment, 
F   (1, 113)=  3.07,  E<.05. 

Reading  Level 

Significant  effects  for  reading   level,   F   (8,222)=  5.88, 
£<.001)   on  the  immediate  test  were  supported  by  univariate  F- 
tests  with  significance  on  all   four  dependent  measures,  F 
(2,113),    E<.0001.     High  readers   (23.07)   and  middle-level  readers 
(20.02)   scored  significantly  higher   than   low  readers   (14.77)  on 
new  fact,    repeated  fact,    new  inference,   and  repeated  inference 
Items,   F   (1,113)=  3.07,   e<.05.     High  readers  scored  significantly 
higher   than  middle-level    readers  on  only  the  repeated  fact  and 
repeated   inference  questions,   F  (1,113)=  3.07,    e.<.05.     This  held 
true  across  practice  conditions. 

Question  Type 

The  difference  on  overall   performance  on  fact  questions 
(10.42)   and  on  inference  questions   (9.32)   was  statistically 
significant,   F  (1,120)=  119.13,   e<.001.     The  correlation  of  Fl 
( immediate  fact  questions)  with  II   ( immediate  inference 
questions)   was  .85. 

I nteract  i  ons 

Multivariate  analysis  of  variance  revealed  no  signl'tcant 
interaction  ©ffactf  for  thm  immediate  test. 
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Table  2 
Mean  Scores  on  Delayed  Test 


Fact  Inference 

Readi  ng 

Level   and           Repeated          New            Repeated          New  Grand 

Cond  1  t  Ion  I  t grrs  I  terns  I  t^ms  I  terns   To ta  1 


High 

Overt 

X 

6. 

59 

5 

.  00 

6. 

76 

5 

35 

23.  71 

SD 

94 

1 

.  70 

1 . 

40 

1 . 

27 

3.80 

Cover  t 

X 

6. 

OS 

5 

.  17 

5. 

17 

3 

92 

20.33 

SD 

1  . 

91 

1 

.  80 

1 . 

99 

1 

38 

6.18 

Summa  r  y 

X 

5. 

15 

5 

.  31 

5. 

62 

5 

92 

22.00 

SD 

1  . 

91 

1 

.  70 

2 . 

C2 

1 

32 

5.64 

Med) urn 

Over  t 

X 

5 . 

41 

.  00 

4  . 

76 

3 

29 

17.47 

SD 

2 . 

OC 

2 

.  C3 

2. 

28 

1 

36 

6.24 

Cover  t 

X 

S . 

00 

O 

.  so 

4  . 

47 

o 

.  ou 

1  c  en 

1  D  .  OU 

SD 

1 . 

96 

2 

.  10 

1  . 

68 

2 

03 

5.93 

Summa  r  y 

X 

ii 

47 

5 

.  34 

4  . 

71 

5 

.  41 

19.82 

SD 

1  . 

70 

1 

.  48 

1  . 

86 

1 

.  58 

4.  42 

Low 

Overt 

X 

3 

91 

3 

.  45 

4  . 

09 

3 

.  00 

14.05 

SD 

2 

77 

1 

.  29 

1  . 

30 

1 

.  90 

4.27 

Cover  t 

X 

50 

3 

.00 

3. 

80 

2 

.  70 

14.  00 

SD 

2. 

55 

2 

.  00 

1 . 

93 

I 

.06 

5.  48 

Summa  r  y 

X 

2 

30 

2 

.60 

2, 

40 

2 

.80 

10.  10 

SD 

I 

25 

.  64 

84 

2 

.  10 

3.87 

Iota  I 

X 

4 

97 

4 

.26 

4. 

80 

4 

.  14 

18.  16 

SD 

2 

20 

1 

.92 

2. 

07 

1 

.92 

6.35 

Practice  Condition 

Read  i  n£ 

Leve  i 

Ques  1 1  on 

Type 

Ovtrt 

IS. 99 

High 

22 

21 

Fact 

9.23 

Cover  t 

17. 

10 

Med  i  urn 

16.  02 

1 nf  erence 

8.94 

Sunmar y 

Id. 

10 

Low 

12 

90 
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Table  3 


Source  Table  for  Univariate  and  Multivariate 
ANOVA  for  Fact  and  Inference  Questions 


Source 
Var I ancp 


I mmed late 
Test 

Prac t  ice 
Cond  i  t Ion 


Read i ng 
Leve  I 


Cond  i  1 1  on 
by  Leve i 


Wi t  h  1  n 

Subject 

Error 


df 


MS 


Unl- 
var 1  a te 
F 


El 


»Mul tl- 
var late 
Approx  imate 
df  F  Ei 


FRl 

2 

20. 

92 

7. 

82 

.  001 

FNl 

2 

17.  92 

7. 

15 

.001 

IRl 

2 

A. 

78 

1  . 

37 

.  259 

IM 

2 

7. 

37 

2. 

76 

.068 

FRl 

2 

54. 

07 

20 

22 

.  001 

FNl 

2 

23. 

86 

9 

52 

.  001 

)r: 

2 

58. 

36 

16 

69 

.  001 

1  Nl 

2 

27. 

91 

10 

^6 

.  001 

FRl 

1 

73 

.65 

.  629 

FNl 

92 

.37 

.  830 

IRl 

36 

.  1 1 

.  980 

INI 

A 

3 

01 

1 

.  12 

.  348 

FRl 

1  13 

2 

.  67 

FNl 

113 

2 

.  51 

IRl 

1  :3 

3 

.  50 

INI 

1  13 

2 

.  67 

e         6.02  .001 


8        5.88  .001 


16         .70  .792 


Pract i  ce 
Condition 


Read  Ing 
Leve  1 


Condi  t Ion 
by  Leve 1 


Within 
Sub  ject 
Error 


FR2 

2 

21 

.  18 

5 

.  51 

.  005 

FN2 

2 

2 

.  lA 

72 

.  A91 

1R2 

2 

10 

.AO 

3 

.  19 

.OAS 

1N2 

2 

16 

.  32 

6 

.  57 

.002 

FR2 

2 

a9 

.67 

12 

.92 

.001 

FN2 

2 

AO 

.  Al 

13 

.  50 

.001 

IR2 

2 

52 

.01 

IS 

.93 

.  001 

1N2 

2 

A3 

.91 

17 

.66 

.001 

FR2 

A 

2.  AS 

.6A 

.637 

FN2 

A 

5.31 

1.77 

.  139 

1R2 

A 

A. 79 

1.  A7 

.217 

1N2 

A 

7.23 

2.91 

.025 

FR2 

113 

3.85 

FN2 

113 

2.99 

IR2 

113 

3.27 

!N2 

113 

2.  A8 

16 


A. 78  .001 


5.62  .001 


1.2A  .231 


•The  Multivariate 


F  value  reported  ii  PlHals'   approxlMte  F. 
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Ho  1  ay  oH  Tggfj^'^tf 


Practice  Condition 

As  shown  in  Table  2  the  mean  scores  for  the  delayed  test 
were  18.99  for  overt,    17.10  for  covert,   and  18.10  for  summary 
groups.     A  significant  effect  was  obtained  for  practice 
condition,   F  (8,222)=  4.78,    e<.001.     The  overt  group  outscored 
the  summary  group  which  in  turn  scored  higher   than  the  covert 
group.     This  delayed  finding  differed  from  the  immediate  test 
results  which  favored  the  summary  group  over  both  the  overt  and 
covert  practice  groups. 

The  analysis  of  performance  by  type  of  question  yielded 
differences  for  repeated   fact  and  new  inference  questions.  On 
repeated  fact   items,    the  mean  scores  of  5.49  for   the  overt 
treatment  and  5.22  for  the  covert  treatment  were  significantly 
higher  than  the  mean  of  4.15  for  the  summary  group,   F  (1,113)= 
3.07,   E.<.05.     Both  practice  groups  also  had  significantly  higher 
scores   than  the  summary  group  for   repeated  fact   items  on  the 
immediate  test. 

On  new  inference   Items,    the  mean  score  of  4.93  for  the 
summary  treatment  was  significantly  higher  ^han  the  mean  scores 
of  4.00  for  the  overt  treatment  and  3.46  fcr   the  covert 
treatment,   F  (1,113)=  3.07,    2.<-05.     Scores  did  not  differ 
significantly  between  groups  on  new   inference   Items  on  the 
immediate  test.     There  ^ere  no  significant   differences  on  the 
delayed  test  on  either  new  fact   items  or  repeated  inference 
items.     On  the   immediate  test,    the  summary  group  had  scored 
higher   than  the  other  two  groups  on  new  fact   items,   but  scores 
did  not  significantly  differ   for  groups  on  repeated  inference 
i  terns . 

Reading  Level 

Reading   level   was  highly  significant   for  the  delayed 
criterion  test,   F   (8,222)=  5.62,    e<.0001  with  all    four  dependent 
measures  significant  at   the   .001    level.     Again,    high  readers 
(22.21)  and  middle-level    readers   (18.02)   scored  significantly 
higher  than   low  readers   (12.90)  on  all    four  .question  types,  F 
(1,113)=  3.07,   2<.0S.     High  readers  scored  significantly  higher 
than  middle-level    readers  on  repeated  fact,    repeated  inference, 
and  new  inference  questions,   F  (1,113)=  3.07,    e.<.05.  This 
finding  differs  from  the   immediate  test  where  no 
significance  between  groups  occurred  on  new  inference  it eras. 

Quest  I  on  Type 

The  difference  on  overall   performance  on   fact  questions 
(9.23)   and   inference  questions   (8.94)  was  again  statistically 
significant,   F  (1,120)=  99.62,   e.<-001.     The  correlation  of 
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student  scores  on  delayed  fact  questions  with  delayed  inference 
questions  was  .67. 

I nt eract  i  ons 

A  significant   Interaction  for   the  delayed  test  was  found  on 
new  inference  items,   F  (4,113)=  2.91,  E<.02. 

higure  1   shows  that   this   interaction  was  primarily  due  to  the 
poor  achievement  of   high  readers   in  the  covert  group. 


Gain  Scores 

Differences  between  the   iinmedlate  and  delayed  test  scores 
were  significant,   F   (1,120)=   139.99,    £<.0Ol.      Univariate  analysis 
were  run  using  gain  scores  for  all   dependent  variables. 
Scheffe's  multiple  comparison  test   found  no  significance  for 
reading   1  eve  1 . 

Significant   effects  were   found   for  practice  condition  or  new 
inference  and  repeated   inference   items.      On  new   Inference  items, 
there  was  a  significant  difference  between  the  summary  condition 
and  the  overt   condition,   F   (1,113)=  3.07,    e<.05.      These  results 
are  difficult   to   Interpret  due  to  the  interaction  effects  found 
for  new  inference  questions.      It   does  appear-    however,    that  this 
significant  gain  score  effect  was  primarily  due  to  the  diffeience 
in  scores  of   the  middle-level   readers  in  these  treatments. 

Significant   differences  between  practice  conditions  were 
found  for  the  overt  and  summary  conditions,    and  the  '"overt  and 
summary  conditions  on   repeated   inference   items,    F   (4*133)=  3.07, 
£^<.05.     This  effect  appears   to  be  due  to  the  difference  in  scores 
of   the  high  readers   in  each  practice  condition. 


Di  scussion 


In  this  St   dy  the  summary  treatment  subjects  displayed 
higher   lev/els  of   recall   on  all   new  fact  and  new  inference  test 
items  than  either  question  treatment  groups.      It  stands  to  reason 
that  what  the  subjects   in  the  summary  condition  remembered  was  a 
function  of  the  cues  provided  for  them  by  the  experimenter  during 
encoding.     Tulving  and  Thompson   (1973)   found  that  the 
effectiveness  of  any  retrieval   cue  was  dependent  on  its  being 
encoded  with  the  material   to  be  remembered.     Summary  subjects 
were  given  paraphrased  concepts  and  cues  which  could  b*  encoded 
siroul tanenusly. 

The  interspersed  questions  required  students  in  the  covert 
and  overt  groups  to  utilize  the  information  they  had  read.  This 
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introduced  a  practice  factor.     Unlike  many  studies   in  the 
literature   (Bourne  &  Haygood,    1959;   Callantine  &  Warren,  1955), 
this  study  found  practice   In  question  form  to  have  only  a  slight 
facilitative  effect  on  achievement.     Covert  and  overt  practice 
groups  were  not  found  to  perform  significantly  better   than  the 
summary  group  that   received  no  questions  or  practice  on  specific 
test   it ems.     A  post  hoc  error  ana  lysis  found  that  overt  treatment 
subjects  who  had  selected  the  wrong  answer  for  any  inserted 
question  in  practice  were  more  apt   to  select   that  same  wrong 
answer  on  the  posttest.     The  correct   response  was  not  selected  on 
the  posttest  even   though   the  students   received   immediate  verbal 
feedbrck   in  practice   indicating  the  correct  response. 

The  overt  subjects  performed  significantly  higher  than  the 
c:overt  subjects  across  all   reading   levels.     Lack  cf  compliance  to 
practice  condition  by  covert  subjects  may  have  accounted   in  part 
for  these  results.      It   is   likely  that   these  results  vary 
accf^rding  to  the  motivation  of   the  students   in  the  covert  group. 

The  results  of   this  study  suggest   that  practice  condition 
and  question  type  may  differentially  affect    learning  and 
retention.     Overt  and  covert  practice  facilitated  recall  of 
repeated  factual    items,   whereas,    summary  subjects  scored 
significantly  higher  on  new  fact  and  new   inference   items.  As 
evidenced  by  total   mean  scores,   allowing  students   to  practice 
factual   test   items   in  treatment   increased  test  scores  on  those 
exact   items.      However,    this  effect  was  not   found  for  inference 
items.     Recall   of  repeated  inference   items  and  other  relevant 
content  was  not  facilitated   through   the  use  of  questions. 

The  Keenan  et  al    (1978)   study   found  that  questions  elicit 
pupil   attention  and  direct   it   to  specific  content.      In  the 
research  done  by  Allen   (1970),    subjects  did  better  on  questions 
relating  to   information  they  were  asked  about   in  adjunct 
questions.      In  this  study,    the  practice  of   specific  fact 
questions  may  have  actually  directed  the  students'   attention  away 
from  other  facts.     Similarly,   practice   in  answering  inference 
questions  did  not  better   prepare  those  subjects   in  the  covert  and 
overt   group  to  answer  new  inference  questions,   but  again  seemed 
to  direct  student  attention  from  relevant  content. 

The  summary  condi  tion  scored  signifi  cant  1 y  hi  gher   than  the 
covert  and  overt   groups  on  new  fact   items.     These  items  were  not 
practiced   in  any  form  by  any  treatment.     Subjects  in  the  covert 
and  overt  question  groups  may  have  been  distracted  from  relevant 
content  by   readi  ng  on  1 y  for  answers   to  ques t  i  ons .     Thi  s  type  of 
depressive  effect  of  questions  on  recall   was  recognized  in  the 
Rickards  and  Denner   (1979)   study.      In  this  study  this  effect  was 
also  found  on  inference  quest i ons .     Repeated  Inference  scores 
were  similar  for  all   treatments,   however,   the  summary  group 
scored  significantly  higher  than  both  question  groups  on  new 
inference  items. 
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The   importance  of  appropriate  placement  of  questions   is  well 
established.     Richards  and  Di  Vista   (1974)   found  that  when 
questions  were  inserted  after  every  two  paragraphs,  meaningful 
learning  questions   led  to  superior  performance.     This  effect 
disappeared  when  questions  were  asked  after  four  paragraphs. 
Results  from  this  study  show  that  questions  asked  after  every 
paragraph  did  not   increase  student  achievement  and  may  have 
interferred  with  processing  and  encoding  of  new  information. 

The   literature  is  far   from  conclusive  about  the  effects  of 
higher    level   questions.     The  nature  of   inference  questions 
requires  that  a  reader  state  a  relationship  between  elements  of  a 
passage.     This   involves  more  complex  cognitive  processing  and  has 
proven  a  more  difficult  task,    especially  for   low  readers.  In 
this  stody,    fact  scores  were  significantly  higher   than  inference 
scores.      However,    inference  questions  had  a  slight  facilitative 
effect   on  retention.     This  was  exhibited  by  the  difference  in 
immediate  and  delayed  test  scores   for   fact  and  inference 
questions.      Inference  question  scores  remained  similar  on  the 
immediate  and  delayed  tests,    whereas,    scores   for   fact  questions 
fell    sharply  on  the  delayed  test.     This   is  consistent  with  the 
Craik  and  Tulving   (1975>   study  where  conceptually  based  items, 
requiring  greater  depth  of  processing,   were  better   retained  than 
the   items  requiring  recall  operations. 

The  summary  group  was  a   highly  effective   treatment.      On  new 
fact   items,    new  inference   items,    and  repeated   inference   items  the 
summary  group  scored  consistently  high.     This  held   true  across 
all    reading   levels   for  both  immediate  and  delayed   tests.  The 
summary  treatment  highlighted  facts  and  concepts   in  narrative 
form  and   reinforced  only  relevant  content.     The  questioning 
treatments,   whether  overtly  or  covertly  practiced,    appear  to  have 
allowed  encoding  of  distractors  and   incorrect  responses. 
Therefore  the  results  of   this  study  would  warn  that  questioning 
strategies  are  beneficial   only  to  the  extent  that  the  correctness 
of   the  response  can  be  controMed   in  a  practice  situation. 

Treatment  times  varied  dramatically  among  the  practice 
conditions.     The  two  question  treatments  were  one  hour  and 
fifteen  minutes   long.     The  summary  treatment  took  only  fifty 
minutes.     With  this  in  mind,   and  comparing  the  overall  immediate 
and  delayed  test  scores,    the  summary  practice  condition  was  the 
most  effective  and  efficient  treatment  for  all    level  readers. 

Reading   level   was  significant  across  treatments.  High 
readers  scored  equally  well   on  fact  and  inference  items.  Medium 
and   low  readers  generally  scored  higher  on  fact  questions  as 
compared  to  inference  questions.     These  results  are  consistent 
with  previous   literature  In  this  area  and  suggest  that  reading 
ability  directly  affects  student  competency  in  answering 
inference  questions. 
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Overall   gain  scores  for  the  three  groups  show  that  overt 
subjects'   scores  tended  to  remain  more  constant  than  those  of  the 
covert  and  summary  subjects.     Although  immt?dlate  test  scores  for 
the  overt   group  were  not  as  high  as  the  summary  group,    the  overt 
participation  appears   to  have  facilitated   long  term  retention. 

On  the  delayed  test  a  practice  by  reading   level  Interaction 
was   found  for  new  in'' ?rence   items.     As  stated  above,  generally 
scores  were   lower  on  the  delayed  test.     The  covert  group  scored 
consistently   lower  on  the  delayed  test  across   reading  levels. 
However,    the  high  readers   in  the  overt   treatment  and  the  medium 
readers  and  high  readers   in  the  summary  treatment  scored 
significantly  higher  on  the  delayed  test  on  new  inference  items. 
The   interaction  appears  to  be  due  to  the  overt  and  summary 
conditions  that  were  especially  effective,   over  time,   for  higher 
readers.     Studies  using  delayed  testing   (Goldbeck  Campbell, 
1962;  Krumboltz  &  Weisman,    1962)   have  shown  overt  treatments  to 
be  more  effective  than  covert  treatments.     However,  little 
previously  has  been  reported  about  summary  treatments. 

Froir  the  immediate  measure  to  the  delayed  measure  repeated 
iteirs   fel  !   more  than  new   items.      It  appears   that  practice  of 
items-   ir.  question  form   leads  to  short   term  retention  of  answers. 
Immediate  results  of   repeated   items  were  higher  due  to  a  practice 
factor.     However,   delayed  test   results  show  that  practice  of 
specific  answe'^s   is  far   less  effective  over  time  than  genera! 
knowledge  of   the  content  itself. 

The  results  of   this  study  suggest  differential  relationships 
between  two  question  treatments  and  the  summary  treatment  with 
respect  to  performance  on  repeated   items  and  performance  on  new 
items.     This   is  of   importance  for   instructional   planning  and 
program  development.     The  present  study  indicates  that   furth  r 
research   in  this  area   is  needed  to  better  determine  the  nature  of 
these  relationships  and   implications  in  regard  to  af f ectiveness 
cf  the  different  practice  conditions  with  respect  to  factual  and 
inferential    learning.     Oral   summaries   should  continue  to  be 
considered  as  viable  alternatives  to  questioning  strategies  and 
deserve  further  investigation. 
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Uses  and  Etfects  of  Learner  Control  of  Context 
and  Instructional  Support  in  Computer-Based  Instruction 

Cognitive  views  regarding  the  effective  teaching  of  mathOTatical  and 
scientific  knowledge  are  increasingly  etrphasizing  the  iirportaiice  of  making 
connections  betwe:en  the  learner's  existing  knowledge  structure  and  new 
information  to  be  learned  (Mayer,  1984;  Peterson,  1988).'  This  basic  principle 
has  been  applied  in  classroom  teaching  through  such  strategies  as  integrating  new 
itaterial  with  class  field  trips  (Wright  &  Stevens,  1983) ,  with  stories  written  by 
the  students  themselves  (Bush  &  Fiala,  1986;  Ferguson  &  Fairbum,  1985;  Hoaner, 
1986;  Leonardi  &  McDonald,  1987)  or  with  interesting  materials  published  in  books 
(Jones,  1983)  and  newspapers  (Daruwalla,  1983) «    It  has  also  been  demonstrated 
numerous  tiroes  in  basic  research  studies.    Ross  and  his  associates  (Ross,  1983; 
Ross,  McCormidc,  &  Krisak,  1983),  for  exairple,  adapted  the  thematic  context  of 
explanations  and  examples  on  a  statistics  lesson  to  college  students'  acadanic 
majors  in  education  or  medical  fields.    Results  consistently  showed  higher 
performance,  particularly  on  transfer  problems,  for  the  adaptive  condition 
relative  to  a  control  group. 

Although  learner  backgrounds  in  the  latter  studies  (e.g.,  Ross,  1983)  were 
well-defined  (education  vs.  medical),  they  are  often  unknown  and  much  more 
diverse  in  real-life  teaching  situations.    Accordingly,  material  that  is 
potentially  meaningful  and  motivating  to  one  student  may  fail  to  be  so  for 
another  given  differences  in  backgrounds  and  current  interests.  Unfortunately, 
the  individual  teacher  will  normally  lack  sufficient  time  and  resources  to 
arrange  appropriate  individualized  options*    Recognizing  these  limitations, 
especially  for  elemaitary  school  students,  Anand  and  Ross  (1987)  used 
microcomputers  to  personalize  mathematical  story  problems  for  children  by 
embedding  information  about  each  child  (e.g.,  hobbies,  friends'  names)  in  the 
problem  context.    Children  who  received  the  personalized  examples  performed 
better  and  reacted  to  the  lesson  more  positively  than  did  those  who  received 
conventional  problems  (also  see  Ross  &  Anand,  1987) .    Despite  these  benefits, 
practical  limitations  of  this  strategy  are  the  time  needed  to  collect  and  input 
personalized  data,  and  the  questionable  appeal  of  the  personalized  contexts  over 
time,  (i.e.,  after  novelty  effects  have  diminished). 

An  inportant  goal  for  instructional  technology  research  is  to  translate  basic 
research  findings  into  aE¥)lied  strategies  that  can  be  implemented  in  classroom 
and  training  settings  (Clark,  1989;  Winn,  1989).    Accordingly,  one  purpose  of  the 
present  research  was  to  develop  a  more  practical  model  for  adapting  context  to 
learner  interests.    Our  specific  approach  used  CBI  to  make  alternative  contexts 
for  sv:atistics  problems  (i.e.,  sports,  business,  education,  no-context)  available 
for  selection  by  individual  learners.    In  this  manner,  adaptations  were 
individualized,  imniediate,  and  applicable  in  feasibility  and  potential  appeal  to 
students  at  a  wide  range  of  grade  levels.    A  second  research  interest  was 
evaluating  the  context  adaptation  strategy  when  used  DOth  ind^)endently  of  and  in 
combination  with  learner  control  (LC)  of  the  number  of  practice  examples. 
Although  LC  is  a  significantly  more  cost-effective  individualization  strategy 
than  developing  "program-control"  models  to  make  selections  for  students  (e.g., 
Hansen,  Ross,  &  Rakow,  1977;  Tennyson  &  Rothen,  1977),  the  question  has  been 
raised  whether  the  typical  student  possesses  sufficient  knowledge  and  interest  to 
make  effective  decisions  (see  reviews  by  Hannafin,  1984;  Ross  &  Morrison,  1989; 
Steinberg,  1977) .    Specifically,  when  a  learning  task  is  relatively  difficult  or 
ability  is  low,  learners  may  be  inclined  to  select  less  instructional  support  so 
they  may  exit  earlier  from  the  task  (Carrier,  Davidson,  &  Williams,  1985;  Ross  & 
Rakow,  1982;  Tennyson,  1980) .    The  present  rationale  was  that  if  the  exaitples 
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were  made  more  meaningful  by  conveying  familiar  crontexts,  motivation  to  use  them 
as  a  learning  resource  should  increase. 

To  investigate  these  questions,  the  present  design  involved  rhe  manipulation 
of  standard  (prescribed)  contexts  versus  LC-context  in  combination  with  standard 
instructional  suj^rt  versus  LC-instructional  support.    One  hypothesis  was  that 
the  LC-context  would  result  in  superior  learning  and  more  positive  task  attitudes 
relative  to  standard  contexts  as  a  result  of  allowing  learners  to  choose  more 
meaningful  topics  that  make  relevant  problem  information  easier  to  trenslate. 
Second,  it  was  predicted  that  when  learners  could  choose  preferred  contexts,  they 
would  be  more  receptive  to  the  material  and  thus  increase  their  selection  of 
instructional  suf^rt.    In  addition  to  testing  these  assumptions,  a  primary  focus 
of  the  study  was  analyzing  the  nature  of  LC  decisions  in  relation  to  the  types  of 
instructional  options  made  available  and  learner  characteristics. 

Method 

Subjects  and  Design 

Subjects  consisted  of  227  undergraduate  students  enrolled  in  required 
(xJucation  courses  at  Memphis  State  University.   They  received  credit  toward  their 
course  grade  for  participating.    Subjects  were  preassigned  at  random  to  15 
treatments  formed  by  crossing  5  types  of  context  (sports,  education,  business, 
no-context,  and  LC)  with  3  conditions  of  instructional  sv:pport  (miniraun,  maximum, 
and  LC) .    Absences  by  several  subjects  wl-io  were  originally  assigned  to  treatments 
resulted  in  slightly  uneven  group  sizes,  ranging  from  13  to  17.    Major  dependent 
variables  consisted  of  achievement  on  three  types  of  posttest  items  (definition, 
calculation,  and  transfer)  and  attitudes  toward  the  lesson*    Achievement  scores 
were  analyzed  via  a  5 (context)  x  3 (support)  MANOVA.    For  attitude  data,  separate 
5x3  ANOVAs  were  performed  on  individual  itans  scores  and  on  the  composite 
score. 

Instructional  Material  and  Treatments 

All  subjects  carpi eted  the  identical  introductory  statistics  unit  presented 
by  computer.    The  unit,  which  was  adapted  from  materials  developed  by  Morrison, 
Ross,  and  O'Dell  (1988),  teaches  principles  of  central  tendency  in  eight  separate 
lessons?  (1)  computing  the  mean  from  sinple  data,  (2)  computing  the  mean  from 
frequency  distributions,  (3)  computing  the  median  with  an  odd  number  of  scores, 
(4)  computing  the  median  with  an  even  number  of  scores,  (5)  determining  the  mode, 
(6)  selecting  the  median  over  the  mean  (Case  I:  open-^nded) ,  (7)  selecting  the 
median  over  the  mean  (Case  II:  extreme  scores),  and  (8)  relative  positions  of  the 
mean  and  the  median  in  skewed  and  symmetrical  distributions.    Each  unit  contained 
an  instructional  section  that  presented  an  explanation  for  the  particular  topic, 
followed  by  four  practice  exaitples.    The  explanatory  sections  were  identical  for 
all  treatments,  but  the  practice  examples  were  intentionally  constructed  to 
coiT'ey  one  of  four  context  variations,  as  described  below. 

Education  contexts  described  situations  involving  teachers  and  students  in 
school  settings,  such  as  scores  on  a  national  spelling  test,  niatiber  of  absences 
from  school,  years  of  experience  of  "Master  Teachers,"  and  IQ  scores.  Sports 
contexts  involved  situations  such  as  free- throw  accuracy  in  basketball, 
strike-outs  in  a  softball  game,  rushing  yards  in  football,  times  in  a  440-yard 
dash,  and  so  on.    Business  contexts  mainly  involved  sales  and  marketing  of 
products,  such  as  the  number  of  magazines  sold  per  week,  number  of  dresses  sold 
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in  a  four-day  period,  repair  rates  for  appliances,  prices  of  coffee  beans,  etc* 
bto-context  probleras  presented  only  the  numerical  values  needed  to  solve  the 
problem,  without  a  sx^Jporting  verbal  context.    Patailei  problems  conprising  the 
four  context  sets  were  identical  in  structure,  numerical  values,  and  in  the 
resultant  numerical  solution.    Subjects  assigned  to  the  four  standard  context 
treatanents  received  the  same  context  on  all  examples.    At  the  beginning  of  each 
lesson,  LC-context  subjects  selected  the  one  they  preferred  from  the  four 
c^>tions.    The  selected  cx^ntext  was  then  presented  on  all-  lesson  examples. 

Instructional  support  variations  involved  the  presentation  of  four  exairples 
on  eadi  lesson  in  the  "maximum"  condition  and  one  exarrple  on  eadi  lesson  in  the 
"minimum"  condition.    In  the  LC  conaition,  subjects  were  presented  with  the  first 
example  and,  after  viewing  it,  were  asked  whether  they  wanted  to  receive  another. 
This  LC  option  was  then  repeated  after  the  second  and  third  examples,  but  not  the 
fourth,  thus  allowing  a  range  of  one  to  four  examples  on  each  of  the  eight 
lessons.    The  various  exaitple  prototypes  were  presented  in  the  same  sequence  in 
all  treatment  variations. 

Instructions  encouraged  students  to  try  to  work  the  problems  on  their  own. 
When  ready,  they  could  press  the  space  bar  to  see  the  solution  st^  and  final 
answer.    They  could  then  back  page  to  study  the  prc*lem  again  or  progress  to  the 
next  problem  or  segm^t.   To  increase  experimaital  control  over  learning 
activities,  the  lessons  and  associated  problems  were  administered  sequentially 
with  no  opportunity  to  skip  any  contoit  or  to  review  old  exanples. 

Instnimentati  on 

Instrunt.*  ,:s  used  in  the  experiment  are  described  below  in  the  order  in  which 
they  were  administered.    The  first  four  measures  consisting  of  the  pretask 
attitude  survey,  reading  test,  computer  attitude  test,  and  unit  pretest,  were 
administered  prior  to  the  experimental  session. 

btelson-Denney  Reading  Test.    Subjects  took  the  "comprehoision  and  rate" 
section  of  Form  D  of  the  Nelson-Denny  Reading  Test  (Btown,  1976) .  Compr^ension 
was  measured  by  having  them  read  eight  paragraphs  and  answer  raultiple-K:hoice 
questions  on  each.    Reading  rate  was  measured  by  asking  them  to  record  the  number 
of  the  line  they  had  reached  lollowing  the  initial  60  seconds  of  reading. 

Unit  pretest.  The  unit  pretest  (paper-and-pencil  mode)  contained  10  items 
on  material  to  be  covered  in  the  instructional  unit.  Five  of  the  items  assessed 
definitional  knowledge,  three  assessed  problem  solving  (calculations) ,  and  two 
assessed  aj^lications.  The  problems  were  parallel  forms  of  those  used  on  the 
posttest,  with  approximately  one-fourth  to  one-^third  of  each  type  represented. 
The  KR-20  internal-consistency  reliability  of  the  pretest  was  determined  to  be 
.76. 

An  independent  pretest  item  asked  subjects  to  assume  that  they  were  about  to 
take  a  mathematics  lesson  in  which  they  could  study  examples  relating  to 
different  themes  (i.e.,  education,  sports,  business,  no-context).    They  were  then 
asked  to  rank  die  four  contexts  in  order  of  preference  ("1"=  most  desirable). 

Unit  posttest.    A  printed  unit  posttest,  adapted  from  Morrison  et  al. 
(1988),  contained  34  items  (total  points  =  43)  organized  into  three  subtests. 
The  knowledge  subtest  (17  items)  assesses  recognition  or  recall  of  definitional 
information  exactly  as  it  appeared  in  the  text.    The  calculation  subtest 
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contained  six  problans  requiring  confutation  of  central  tendency  measures  frcxn 
new  data  not  ased  in  lesson  exarnples.    The  transfer  subtest  consisted  of  seven 
problems  that  involved  interpreting  and  explaining  the  rationale  for  how  c^tral 
tendaxy  would  vary  with  changes  in  distributions  or  individual  scores.    Items  of 
thic  type  were  not  included  in  the  lesson,  nor  were  the  underlying  principles 
explicitly  stated.    On  all  transfer  items,  there  was  essentially  one  correct 
explanation  for  the  effect  in  questiono    Scoring,  which  was  done  by  two  of  the 
authors/  was  therefore  straightforward  in  nearly  all  cases.   Where  an  answer  was 
ambiguous  or  partially  correct,  the  two  scorers  examined  it  jointly  and  reached  a 
mutual  scoring  decision. 

Scoring  rules  on  objective  items  and  calculation  problems  awarded  one  point 
for  a  correct  answer.    On  interpretative  items,  one  point  was  awarded  for  a 
correct  answer  and  an  additional  point  for  a  correct  eKplanation.    The  total  test 
KR-20  reliability  was  .90.    Transfer  subtest  reliability  was  .84. 

Task  attitude  survey.    A  survey  consisting  of  six  Likert-type  items  was 
used  to  assess  subjects'  reactions  to  the  task  and  materials. 
Intemal-oxisistaicy  reliability  was  .54.    LC-context  subjects  reacted  to  two 
additional  items,  one  asking  whether  examples  with  familiar  themes  were  easier  to 
learn,  ai>d  the  other  v^ether  receiving  a  mixture  of  themes  is  preferable  to 
experiencing  only  one  theme.    They  were  also  asked  open-ended  questions, 
regarding  the  desirability  of  being  able  to  select  contexts  and  to  learn  from  a 
cortputer. 

Results 

Achievement 

Posttest  achievement  was  analyzed  via  a  5 (context)  x  3 (instructional  support) 
MANOVA  with  the  knowledge,  calculation,  and  conceptual  subtests  as  dependent 
variables.    None  of  the  sources  of  variance  was  significant.    Examination  of 
means,  as  shown  in  Table  1,  showed  little  difference  between  treatments,  with  10 
out  of  the  15  treatment  means  being  within  1.5  points  of  the  overall  sample  mean 
of  27.45  (64%  correct). 


Insert  Table  1  about  here 


Attitudes  and  completion  time.    Analyses  of  individual  attitude  items  on 
the  follow-up  survey  and  total  score  showed  only  one  effect.    As  might  be 
expected,  subjects  who  received  minimum  support  found  the  lesson  faster  moving 
(  p  <  .04)  than  those  who  received  maximum  support.    Overall,  most  itenis  evoked 
fairly  mixed  reactions  with  the  median  response  falling  between  "undecided"  and 
"agree."   The  lowest  mean  ratings  on  the  six  core  items  (items  received  by  all 
subjects)  were  for  "the  lesson  moved  quickly"  and  "I  prefer  this  method  of 
learning  over  lecture."   The  highest  mean  ratings  were  for  "sufficiency  of 
instruction"  and  the  "readability  of  the  layout."    Interestingly,  the  most 
favorable  reactions  on  thd  entire  survey  occurred  on  the  LC  item  concerning  the 
desirability  of  learning  fron  familiar  theines  (  M  =  4.11),  with  89%  of  the  sample 
indicating  either  stijong  agreement  (25%)  or  agreement  (64%).    Only  41%  agreed  or 
strongly  agreed,  however,  that  they  would  prefer  a  mixture  of  themes  rather  than 
the  same  theme  throughout,  but  this  response  did  not  correlate  with  actual 
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teirlencies  to  change  themes  during  instruction.    Finally,  the  two-way  ANOVA  on 
conpletion  tiiw^  showed  the  instructional  suroort  main  effect  to  be  the  only 
significant  result  (  £  <  .OB).    As  would  be  expected,  tlie  ordering  of  treatirtents 
from  most  to  least  time  was  maximum  (  M  -  30*0  min.) ,  learner-control  (  M  = 
26*7) ,  and  minimum  (  M  =^  25*4)  • 

Open-Ended  Attitude  Responses 

Qualitative  analyses  of  open-ended  responses  regarding  the  desirability  of 
selecting  problem  themes  showed  92%  to  be  positive,  5%  neutral,  and  only  3% 
negative.    Follow-up  analyses  of  the  positive  response^  indicated  that  the  most 
frequently  given  rationale  (38%)  specifically  mentioned  the  advantages  of  context 
selection  for  relating  the  material  to  existing  knowledge  or  experiences. 
Exait¥>les  were:  "Wonderful,  I  was  able  to  relate  it  (the  materials)  to  something  I 
understood,"  and  "It  was  good  to  see  that  central  tendency  can  be  applied  to 
everyday  living  situations."   Other,  lesser  used  response  cate^oties  consisted  of 
"General"  (37%;  "I  liked  it"),  "Interest"  (16%;  "The  context  inade  the  problems 
more  interesting"),  "Familiarity"  (11%;  "I  am  sports  minded,  so  I  felt  more 
comfortable  with  sports"),  and  "Variety"  (5%;  "I  liked  choosing  the  themes 
because  I  was  able  to  add  variety"). 

In  response  to  the  question  about  the  desirability  of  learning  math  by 
conputer,  the  responses  were  mixed,  with  58%  classified  as  positive,  14%  as 
neutral,  and  28%  as  negative.    The  most  frequently  mentioned  positive  aspects 
were  the  self-pacing  and  feedback  components.    Most  frequently  mentioned  as  a 
negative  factor  was  the  lack  of  interaction  with  a  human  teadier. 

Learner  Control  Outcomes 

Additional  analyses  examined  LC-context  and  LC-instructional  support  outcomes 
and  their  relationship  with  individual  difference  variables  consisting  of  gender, 
pretest  score,  task  attitude  score,  reading  ability,  and  total  posttest  score. 

Context  rankings  and  preferences.    On  the  pretest,  subjects  were  asked  to 
rank  the  four  contexts  in  order  of  preference.    As  shown  in  Table  2,  the  ordering 
of  mean  rankings  (lower  mean  =  higher  preference)  and  of  frequencies  with  which 
the  contexts  were  ranked  first  was  education  highest,  followed  by  sports, 
business,  and  no-context.    Statistical  conparison  of  number  one  rankings  across 
contexts  indicated  a  highly  significant  difference,  (  £  <  .001).  Gender 
differences  were  also  highly  significant,  (  £  <  .001):  males  were  more  likely  to 
rank  sports  highest  (66%)  than  were  females  (14%) ,  while  the  converse  occurred 
for  education  (17%  vs.  52%) . 


Insert  Table  2  about  here 


Also,  as  shown  in  Table  2,  the  relative  frequencies  with  which  contexts  were 
actually  selected  across  the  eight  lessons  were  ordered  similarly  to  initial 
rankings  with  the  most  popular  choice,  education,  being  selected  on  almost  half 
of  the  eight  trials  (  M  =  3.66) ,  followed  by  sports  (  M  =  1.88) ,  no-context  (  M  = 
1.61),  and  business  (  M  «  .91).    Comparison  of  the  fouT  context  selection  scores, 
using  the  nonparametric  Freidman  ANOVA  By  Rank  test  (Hays,  1981) ,  were 
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significant^  (3)  =  19,86,  £  <  .001,  thus  cxDnfirming  the  observed  ptoference 
differential.    Analyses  of  gender  differences  were  significant  on  three  of  the 
contexts;   males  were  more  likely  than  females  to  select  spor*:s  (  £  <  .05) ,  but 
less  likely  to  select  no-context  (  £  <  .01)  and  education  (  £  <  .001)  • 
Correlations  betwera  pretest  scores  and  selection  total?  '*i):ther  revealed  that 
higher  prior  achievement  was  associated  with  a  greater  t-*'    rv^y  to  select  the 
no-context  option,  £  (44)  =  .32,  £  <  ,05,  and  a  lesser  fce  *ency  to  select  the 
education  context,  £  (44)  =  -.47,  £  <  ,001. 

Context  selection  patterns.    Variations  in    ^ntext  preferences  during 
learning  were  examined  by  tabulating  the  number  of  times  selections  were  ciianged 
across  adjacent  lessons  (maximum  score  =  7)  •    Results  showed  that  39%  of  the 
subjects  selected  the  same  context  on  all  lessons,  25%  made  1  or  2  changes,  and 
36%  made  3  or  more  changes.    Fourteen  percent  of  the  total  group  sattpled  all  four 
options,  and  25%  examined  at  least  three  out  of  four.    Change  scores  were  not 
found  to  be  correlated  with  any  of  the  individual  difference  variables. 

Other  results  rc-vealed  that  differ«icss  in  the  relative  frequeaicies  v;ith 
which  the  four  contexts  were  sel'xrted  early  (Lesson  1)  r  spared  to  late  (Lesson 
8)  in  the  task  were  not  significant.    Students  who  s^^^-cted  the  no-context  option 
had  significantly  higher  pretest  scores  than  those  who  selected  the  education 
context  on  both  Lesson  1  [  M  =  64%  correct  vs.  M  =  21%  correct,  t(25)  «  3«72,  £  < 
.001]  and  Lesson  8  [  M  =  45%  vs.  M  =  19%,  t(27)  =  2.54,  £  <  .05], 

Instructional  support  selections.    The  total  quantity  of  examples  selected 
across  the  8  lessons  ranged  from  8  to  32,  with  a  standard  deviation  of  6.5.  The 
distribution  was  positively  skewed,  with  a  mean  of  15.5  (1.9  per  lesson) ,  median 
of  13.5,  and  mode  of  11.0.    As  expected,  total  quantity  selected  was  negatively 
correlated  with  pretest  scores,  £  (75)  =  -.26,  £  <  5. 

Examination  of  the  total  quantity  of  examples  selected  by  each  context  group 
shewed  that  the  means  were  ordered  in  the  predicted  direction  with  the  no-context 
mean  lowest  (  M  =  12.0)  and  the  LC  mean  (  M  =  18.5)  highest.    Fairly  comparable 
means  were  obtained  for  the  three  thematic  contexts,  education  (  M  =  16.0) , 
sports  (  M  =  16.7),  and  business  (  M  =  14.6).    For  purposes  of  analysis,  the  data 
for  the  three  thematic  context  groups  were  pooled,  resulting  in  a  three  treatment 
group  ANOVA  design;  standard  thematic  context,  no-context,  and  LC.    The  treatment 
effect  was  significant,  F(2,72)  =  4,21,  £  <  ,02.    Follow-up  corrparison  of 
means,  using  the  Tukey-HSD  procedure,  showed  that  LC  subjects  selected 
significantly  more  (  £  <  .05)  examples  than  did  no-context  subjects,  but  not  more 
than  subjects  reviewing  thematic  contexts. 

Support  selection  patterns.    Tabulations  of  the  number  of  times  subjects 
varied  their  selections  of  examples  across  adjacent  lessons  (maximum  score  =  7.0) 
revealed  an  overall  mean  of  2.63  (a  38%  rate).    Overall,  91%  of  the  saitple  varied 
their  support  selections  at  least  one  time.    Three  individual  differoice 
variables  were  related  to  these  tendencies.    Selection  changes  were  more  frequent 
for  females  (  M  =  2.87)  than  for  males  (  M  =  1.33),  t(73)  =  3.08,  £  <  .001;  for 
slower  readers  than  for  faster  readers,  £  (75)  =  -.24,  £  <  .05;  and  for  higher 
posttest  performers,  £  (75)  =  .24,  £  <  .02. 

Correlations  Between  Learner  Variables  and  Experimental  Outcomes 

Correlations  between  learner  charact<5ristics  and  task  outcomes  generally 
revealed  the  expected  tendencies  for  learners  with  higher  prior  achievement 
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(pretest  scores)  and  reading  ability  to  complete  the  task  faster,  score  higher  on 
tha  posttest,  and  view  the  task  itiore  favorably*    To  examine  these  relations  from 
a  multivariate  perspective,  separate  multiple  regression  analyses,  using 
step-wise  entry  of  variables,  were  conducted  on  each  of  the  three  criterion 
variables:    posttest,  attitudes,  and  ciine*    Each  criterion  variable  was  entered 
as  a  predictor  for  the  other  two,  along  with  pretest  score,  total  number  of 
examples  received,  reading  rate,  and  reading  conprdiension.    Results  for  the 
postte:^^  showed  significant  predictors,  in  order  of  entry,  to  be  pretest, 
mathematics  attitude.s,  and  reading  compr^ension  (  R  ^  =  .35,  d  <  .001);  for 
task  attitudes,  predictors  were  pretest  and  reading  rate  (  R        .08,  £  < 
.001);  for  conpletion  time,  they  were  number  of  examples,  pretest,  and  reading 
conprehension  (  R  ^  =  .15,  £  <  .001)  • 

Discussion 

The  achievement  results  of  this  study  failed  to  support  the  hypothesized 
benefits  for  learning  of  allowing  learners  to  select  preferred  contexts  for 
practice  exanples.    However,  findings  regarding  selections  of  instructional 
support  and  context  in  LC  conditions  provided  some  interesting  insights  into  how 
learner  control  is  used  by  different  types  of  stud«its  as  well  as  suggestions  for 
increasing  its  effectiveness  as  a  CBI  strategy*    Also,  ccnpared  to  earlier 
applications  (Ross,  1983;  Anand  &  Ross,  1987) ,  a  practical  model  for 
incorporating  contextual  adaptation  in  CBT  was  developed  and  demonstrated* 

With  regard  to  achievement,  the  instructional  ;naterial  appears  to  have  been 
scmewhat  difficult  for  students  given  the  scop?  and  conditions  of  the  learning 
task.    In  the  regular  self -paced  statistics  course  from  which  the  present  lesson 
»^s  adapted,  students  generally  take  two  or  three  days  (presumably  involving  at 
least  several  hours  of  self -study)  to  complete  essentially  the  same  unit  of 
material  and  average  approximately  89%  on  a  comparable  test  compared  to  the 
present  average  of  64%.    Accordingly,  as  suggested  by  the  multiple  regression 
data,  adiievament  was  strongly  related  to  subjects*  abilities  and  prior  learning 
(pretest,  math  attitudes,  reading  ability),  but  generally  inclependent  of  task 
variables.    Even  the  amount  of  instructional  support  received,  a  highly 
influential  variable  in  most  learning  situations  (Carrier  &  Williams,  1988;  Ross 
&  Rakow,  1982;  Tennyson  &  Rothen,  1977) ,  had  no  impact  on  performance.  An 
additional  factor  that  might  have  attenuated  contextual  effects  compared  to 
previous  studies  (Ross,  1983;  Ross,  et  al.,  1986)  was  the  restriction  of  the 
contextual  manipulations  to  practice  examples  only,  after  the  relevant  principles 
and  operations  had  been  taught.    Also,  the  simplicity  of  the  contexts  with  regard 
to  length  and  detail  probably  reduced  the  meaningfulness  and  thus  the  interest 
value  of  the  applications  conveyed. 

Despite  the  absence  of  performance  effects,  the  learner  control  outcontes  were 
revealing  regarding  common  selection  strategies  employed  and  their  relationship 
to  learner  differences.    Consistent  with  previous  studies  (Carrier  et  al.,  1985; 
Ross  &  Rakow,  1982;  Tennyson,  1980) ,  there  was  a  general  tendency  by  subjects  to 
select  minimal  instructional  support  (less  than  two  exanples  per  rule).  Given 
the  generally  low  posttest  scores,  this  tendency  may  reflect  subjects*  inability 
to  gain  an  adequate  understanding  of  the  content  from  the  examples.    As  Clark 
(1984)  suggests,  students  tend  to  avoid  high  suKX>rt  options  (and  the  extra 
effort  entailed)  when  they  expect  to  fail  anyway.    Another  possible  explanation 
is  that,  as  a  result  of  working  in  an  experimental  rather  than  actual  learning 
setting,  subjects  may  have  had  low  motivation  to  prolong  the  task  by  selecting 
additional  support. 
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Identifying  effective  uses  of  learner  control  has  relevance  to  the  practical 
problem  of  making  CBI  more  adaptive  for  learners*    Program-<x>ntrolled  adaptive 
strategies  that  systeroatically  select  learning  resources  for  students  can  provide 
powerful  instructional  adaptations  (e.g.,  Tennyson  &  Rothen,  1977;  Park  & 
Tennyson,  1986) ,  but  are  very  costly  to  develop  due  to  the  sophistication  of  the 
executive  models  needed  to  analyze  performance  and  generate  individualized 
prescriptions  oriented  to  the  learning  task  concerned*    Another  alternative  is 
"advisanait"  or  "coaching"  in  which  students  are  given  direction  by  the  program 
about  what  resources  to  select,  but  are  ultimately  free  to  make  their  own  choices 
(Tennyson  &  Buttrey,  1980) .   But  to  provide  such  advice,  the  program  must  sannehow 
determine  what  is  "adaptive"  for  the  individual,  a  process  that  carries  similar 
executive  modeling  requirements  (and  associated  developtmait  costs)  as  program 
control. 

A  third,  more  practical  alternative  is  to  identify  strategies  that  facilitate 
independent  adaptive  decision-making  by  learners.    Suggestive  evidence  from  the 
present  study  is  that  when  learners  could  select  problems  themes  that  interested 
than,  the  number  of  examples  they  elected  to  examine  significantly  increased 
compared  to  the  no-KX)ntext  group  (a  54%  increase)  and  was  direct ionally  higher 
compared  to  the  combined  thematic  context  groups  (a  17%  increase)  •    This  basic 
idea  is  consistent  with  current  theoretical  views  regarding  uses  of  interesting 
displays  and  subject  matter  to  increase  motivation  and  attrition  in  CBI  (Keller  & 
Suzuki,  1988).    It  was  also  supported  by  subjects'  attitude  ratings  and 
open-ended  survey  responses,  with  approximately  90%  of  the  LC  sample  comnenting 
on  the  motivational  and/or  learning  advantages  of  dioosing  preferred  prd)lem 
themes*    Such  personalized  properties  can  increase  the  motivational  effects  that 
learner  control  in  general  seems  to  have  for  students  as  a  CBI  design  roraponent 
(Kinzie  &  Sullivan,  1989) • 

The  nutnber  of  shifts  in  the  quantity  of  examples  selected  across  lessons  was 
the  only  task  outcome  variable  to  relate  significantly  to  posttest  achievement* 
This  correlational  outcome  could  have  been  influenced  by  nunerous  intervening 
variables.    One  possiblity,  however,  is  that  subjects  who  varied  their  selections 
more  frequently  were  making  greater  use  of  metacognitive  strategies  in  assessing 
their  needs  on  the  lesson  (Tobias,  1987).    This  interpretation  suggests  the 
possible  benefits  of  orienting  LC  strategies  to  help  students  to  gain  greater 
awareness  of  their  needs  and  appropriate  study  behaviors.    An  exenplary  approach 
is  to  require  LC  students  to  r^)eat  formative  test  items  that  they  initially 
answer  incorrectly  (Kinzie,  Sullivan,  &  Berdel,  1988).    Consequently,  they 
receive  imttvediate  confirmation  of  their  errors,  while  knowing  that  they  will  be 
held  accountable  for  learning  the  correct  information  before  progressing  on  the 
task.    As  an  alternative,  explicit  on-task  advisement  (e.g..  Carrier,  Davidson, 
Williams,  &  Kalweit,  1986)  might  be  used  to  pratpt  students  to  reflect  on  their 
degree  of  understanding  and  to  vary  their  selection  of  LC  resources  accordingly 
(e.g.,  "Are  you  understanding  this  section?   Consider  selecting  additional 
examples  if  you  are  having  any  difficulty.").    This  type  of  advisaptent,  though 
requiring  additional  empirical  validation,  constitutes  a  highly  practical  means 
of  providing  adaptive  coaching  in  CBI  lessons. 

The  present  study  is  also  one  of  several  recent  CBI  studies  (i.e..  Carrier  & 
Williams,  1988;  Kinzie  et  al.,  1988;  Ross,  Morrison,  &  O'Dell,  1988)  that  have 
identified  reading  ability  as  a  relatively  strong  predictor  of  achievement. 
Although  reading  skills  should  certainly  affect  text  comprehension  from  any 
presentation  medium,  the  unique  conditions  and  constraints  of  CBI  text  displays 
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(the  small  display  area^  keypressing  requirements/  special  forma tS/  back-  and 
forward-paging  limitations,  etc.)  may  exacerbate  attentional  or  comprehension 
problems' for  the  low-ability  reader  "(see  Hepner,  Anderson,  Farstrup,  &  Weiderman, 
1985) .    Further  research  should  focus  on  this  issue  and  the  possibility  of 
designing  practical  LC  options  that  allow  individuals  to  vary  text  formats  or 
content  (e.g.,  as  in  Morrison  et  al.,  1988)  according  to  preferences  or  needs. 
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Table  1 

Total  Posttest  Mean  Percentage  Correct  for  Context  x  Instructional  Support 
Treatments 


Context 


Minimum 


Instructional  Support 


Maxiimm 


Learner-Control 


Education 

Sports 

Business 

No-Context 

Learner-Control 


30.6 
24.0 
26.3 
27.5 
23.8 


24.4 
28.8 
24.2 
27.5 
27.0 


25.9 
28.1 
28.4 
27.0 
27.8 


Note:    Possible  range  of  scores  was  0-43 


Table  2 

Sunmary  of  Learner  Control  Context  Rankings  and  Selections 


Initial  Rankings  Lesson  Selections 


Overall 

Male 

Female 

Contexts 

M 

Rfl 

M 

M 

M 

Education 

1.70 

.55 

3.66 

1.00 

4.07 

Sports 

2.56 

.24 

1.88 

4.33 

1.42 

Business 

2.74 

.11 

.91 

2.38 

.68 

No-Context 

2.99 

.11 

1.61 

.33 

1.82 

^  Relative  frequencies  of  #1  rankings. 

Means  indicate  average  number  of  selections  across  the  8  lessons. 
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Abstract 


The  purpose  of  this  study  is  to  determine  an  optimal  schedule  for 
the  frequency  and  spacing  of  drill  and  practice  in  the  learning 
of  verbal  information.    Subjects  for  phase  1  were  sixty-two  4th, 
5th  and  6th  graders  in  a  public  elementary  school.    Using  a 
computer-based  drill  which  incorporated  in  its  structure  the 
findings  of  past  reseairch,  the  students  each  determined  their  own 
practice  schedule  over  a  period  of  two  weeks.    A  pretest  and 
posttest  were  used  to  measure  the  level  of  success  of  each 
student  in  learning  the  assigned  items.    Results  suggest  that  two 
times  through  the  optimized  drill,  one  near  the  time  of  original 
learning  and  the  other  near  the  retention  measure,  are  sufficient 
for  maximal  learning  of  the  drill  material.    This  conclusion  will 
be  tested  in  phase  2  of  the  study. 
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Frequency  and  Spacing  of  Drill  and  Practice 
in  the  Learning  of  Verbal  Infointiation 
Using  a  Computer-Based  Drill 


The  purpose  of  this  study  is  to  determine  an  optimal 
schedule  for  the  frequency  and  spacing  of  drill  and  practice  in 
the  learning  of  verbal  information.    In  order  to  provide  an 
optimal  practice  tool,  the  study  is  using  a  computer-based  drill 
which  incorporates  in  its  structure  the  findings  of  past 
research. 

Since  computers  entered  the  educational  realm ,  drill  and 
practice  programs  have  been  widely  used  (Gerlach,  1984;  Siegel  & 
Mi^jselt,  1984;  Davidson  &  Traylor,  1987).     Some  have  considered 
drill  and  practice  a  poor  use  of  the  computer  because  it  does  not 
use  many  of  the  capabilities  of  the  machine  (Yates,  1983;  Fuson  & 
Brinko,  1985;  Salisbury,  1985).    Research  has  been  conducted  to 
compare  computer  drill  and  practice  programs  with  more 
traditional  methods.    Results  often  implied  that  the  computer  and 
traditional  drills  yielded  the  same  results  (Fuson  &  Brinko; 
Jamison,  Suppss  &  Wells,  1974;  Campbell,  Peck,  Horn  &  Leigh, 
1987).    In  these  studies,  the  traditional  drill  was  often  made  as 
alike  to  the  computer  drill  as  possible.     This  suggests  that  the 
medium  itself  does  not  make  a  significant  difference  in  the 
effectiveness  of  drill  and  practice. 

At  the  same  time,  performance  gains  and  reduced  amount  of 
time  were  found  in  several  studies  (Kearsley,  Hunter  &  Seidel, 
1983;  Kulik,  Bangert  &  Williams,  1983;  Suydam,  1984;  Carrier, 
Post  &  Heck,  1985) .     Some  studies  also  observed  improved 
attitudes  (Kulik  et  al;  Chambers  &  Sprecher,  1984).    The  most 
consistent  positive  results  of  improvement  in  achievement  were 
found  for  the  learning  disabled  (Edwards,  Norton,  Taylor,  Weiss  & 
Dusseldorp,  1975;  Jamison  et  al;  Chambers  &  Sprecher;  Suydam; 
Balajthy,  1984;  Chadwick  &  Watson,  1986). 

Over-all,  computer  based  instruction  has  spurred  research 
throughout  the  field  of  instruction  (Kearsley  et  al,  1983) . 
Characteristics  of  learners  and  how  these  affect  the  learning 
process  are  being  studied.    The  computer's  ability  to  respond  to 
individual  differences  and  to  help  to  analyze  them  is  being  more 
fully  realized.     For  drill  and  practice  programs,  aspects  of 
individualization  are  accomplished  through  processes  such  as 
feedback,  correction  of  errors,  varying  levels  of  difficulty,  and 
maintenance  of  individual  student  records.     For  example,  by 
keeping  track  of  errors,  the  computer  can  strategically 
reintroduce  missed  items  in  order  to  maximize  the  probability  of 
retention.     Some  of  these  features  would  be  difficult  to 
implement  with  more  conventional  means  such  as  worksheets  or 
teacher  pr'^sentation.    The  computer-based  research  of  today, 
then,  is  built  upon  results  of  earlier  studies  involving 
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instructional  methods  administered  with  more  traditional  means. 
With  ths  computer,  however,  more  complex  strategies  can  be  used 
than  in  the  past* 

Some  research  has  been  conducted  to  study  various  features 
of  drill  and  practice  in  order  to  determine  what  methods/ 
qualities  are  most  effective.    Among  the  topics  researched  are 
spacing  and  length  of  practice  sessions,  review  of  missed  items, 
and  type  of  feedback* 

spacing  and  Length  of  Practice  Sessions 

Reynolds  and  Glaser  (1964)  conducted  research  on  the  effects 
of  repetition  and  spaced  review  in  the  learning  of  a  complex 
task.    The  study  demonstrated  the  relative  ineffectiveness  of 
massed  practice  following  initial  learning.    There  has  been  much 
evidence  to  suggest  that  short,  spaced  practice  periods  yield 
better  results  than  long  concentrated  practice  periods  (Anderson, 
1980) .    At  the  occasion  of  each  practice  period,  the  learning 
context  is  somewhat  different  (e.g.  student's  pre-knowledge  of 
the  material,  time  of  day,'  surrounding  conditions,  etc.),  causing 
the  information  to  be  encoded  somewhat  differently  each  time 
(Salisbury,  1984) . 

According  to  McGeoch  (1932),  forgetting  is  as  much  a 
function  of  the  amount  of  time  between  use  of  the  material  as  of 
the  experiences  which  occur  during  the  intervening  period.  Thus 
the  optimal  spacing  of  practice  sessions  would  depend  on  many 
factors.    Research  has  shown  that  spaced  repetition  does  increase 
meaningful  retention  (Peterson,  Wampler,  Kirkpatrick  &  Saltzman, 
1963;  Ausubel  &  Youssef,  1965). 

Gay  (1973)  studied  the  effect  of  the  temporal  position  of 
reviews  on  the  retention  of  mathematical  rules.     She  experimented 
with  placing  a  review  either  1  day,  1  week,  or  2  weeks  after 
original  learning.    A  test  administered  3  weeks  after  original 
learning  showed  that  temporal  position  was  not  a  significant 
factor,  though  the  retention  of  all  review  groups  was 
significantly  more  than  a  no-review  group.    A  second  experiment 
compared  two  reviews,  placing  them  1  and  2,  1  and  7,  or  6  and  7 
days  after  original  learning.    Again,  all  groups  retained  more 
than  a  no-review  group,    she  concluded  that  additional  reviews 
are  highly  effective  in  terms  of  delayed  retention,  and  are  very 
efficient  in  terms  of  numbers  of  examples  as  well  as  amount  of 
time  spent  on  those  examples.     Peterson,  Ellis,  Toohill,  and 
Kloess  (1935)  had  shown  that  a  second  review  is  more  effective 
than  the  first,  and  Gay's  study  found  that  one  c^arly  and  one  late 
are  better  than  two  early  and  two  la+'.e  rr^iews;  i.e.  the  reviews 
of  1  and  7  days  after  original  learning  resulted  in  significantly 
better  retention  than  the  reviews  of  1  and  2  days  after  learning. 

Merrill  and  Salisbury  (1984)  summarize  research  as 
indicating  that  several  short  spaced  reviews  are  more  effective 
than  a  few  massed  reviews.     Goldenberg  (1987)  suggests  studying 
this  question  of  spaced  practice,  added  to  his  optimized  review 
schedule  (explained  in  the  next  section) ,  taking  into  account 
individual  learner  differences. 
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Review  of  Missed  Items 

When  drilling  verbal  information ,  the  learner  can  make 
errors.     It  is  then  necessary  to  review  those  items  until  they 
are  learned  correctly.    Research  has  examined  the  question  of  how 
often  and  how  soon  to  reintroduce  a  missed  item. 

Siegel  and  Misselt  (1984)  conducted  a  study  involving  a 
drill  program  on  English- Japanese  (transliterated)  word  pairs  in 
whi'-'h  an  increasing  ratio  review  was  used.     Their  results  suggest 
that  increasing  ratio  review  techniques  (e.g.  after  1/3,5 
intervening  items)  are  more  effective  than  traditional  techniques 
(e.g.  1  later)  or  no  review  at  all.     They  also  concluded  that  a 
1-later  review  is  no  more  effective  than  no  review,  suggesting 
that  a  2-3-5  or  2-4-6  review  schedule  may  be  more  effective  than 
a  1-3-5  review  schedule. 

Goldenberg  (1987)  studied  the  optimum  number  of  intervening 
items  that  should  be  inserted  after  an  incorrect  response  before 
a  drop-off  in  memory  occurs.    He  used  the  same  subject  material 
as  Siegel  and  Misselt  (1984) .     He  concluded  that  the  optimum 
position  was  after  two  intervening  items,  with  the  second 
recommended  position  being  after  three  intervening  items. 
Salisbury  (1985)  suggests  that  missed  items  reappear  according  to 
a  spaced  review  schedule  of  up  to  three  review  positions. 

Feedback 

The  study  of  Siegel  and  Misselt  (1984) ,  discussed  in  the 
last  section,  also  used  adaptive  feedback  techniques  with 
discrimination  training.    This  significantly  reduced  the  number 
of  discrimination  errors  on  the  post -test.     In  their  feedback 
paradigm,  the  types  of  mistakes  were  divided  into  two  types  -  an 
"out-of-list"  error,  when  the  learner's  response  is  not  an  answer 
to  another  item  in  the  drill  list,  and  a  "discrimination"  error, 
when  the  learner  types  a  response  that  is  an  answer  to  another 
item  in  the  drill  list.    When  the  first  type  of  error  is  made, 
the  correct  answer  is  provided.     In  the  latter  types, 
discrimination  feedback  is  provided;  i.e.  the  learner  is  informed 
of  the  correct  response  as  well  as  what  stimulus  was  responded 
to.     For  example,  in  a  drill  on  the  states  and  capitals,  if  the 
student  types  St*  Paul  (the  answer  to  the  stimulus  Minnesota) 
instead  of  Jefferson  (the  answer  for  the  stimulus  Missouri) ,  the 
feedback  provided  is  "the  capital  of  Missouri  is  Jefferson;  St, 
Paul  is  the  capital  of  Minnesota." 

Salisbury  (1985),  in  describing  what  he  considers  "good" 
drill  and  practice  strategies,  incl.ides  discrimination  training 
as  one  of  the  features  of  the  strategy. 

Observation 

The  many  student  differences  which  can  affect  success  in 
various  learning  strategies  can  make  it  difficult  to  come  to 
clear-cut  conclusions  regarding  aspects  of  computer-based 
instruction.     Tobias  (1981),  in  a  discussion  of  adapting 
instruction  to  individual  difference  among  students,  suggests 
that  research  might  be  begun  by  observation  of  wh?t  students  of 
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varying  individual  difference  characteristics  actually  do,  and 
what  instructional  options  they  select •    '^'rom  this  observation, 
the  researcher  can  then  generate  a  prediction  as  to  what  strategy 
might  be  best  for  them. 

Observation  is  the  method  used  in  phase  1  of  this  study.  In 
order  to  first  obtain  additional  data  from  which  to  propose  and 
test  a  review  schedule,  the  study  is  being  carried  out  in  two 
phases.    The  data  obtained  through  observation  in  phase  1 
(already  completed)  is  being  used  to  determine  a  schedule  which 
will  then  be  tested  in  phase  2.    This  report  includes  the  results 
from  phase  1. 

Methods 

Sample 

Students  involved  in  phase  1  of  this  study  were  the  sixty- 
two  4th,  5th  and  6th  graders  in  an  urban  Minneapolis  public 
elementary  school.      Absenteeism  was  the  major  reason  for  a 
student's  not  being  able  to  participate  fully  in  the  study.  55 
students  (89%)  took  the  pretest,  58  (94%)  used  the  computer 
drill,  and  61  (98%)  took  the  posttest.     51  students  (82%) 
participated  in  all  three  parts  of  the  study. 

Data  was  obtained  from  the  school  files  on  CAT  exams  taken 
in  the  fall  of  1988  by  58  (94%)  of  these  svudents.     Math  concepts 
and  math  computation  results  were  averaged  to  obtain  a  math 
score.    The  group  mean  for  math  is  66.1  with  a  standard  deviation 
of  25.8.    The  mean  for  reading  is  67.6  with  a  standard  deviation 
of  26.8.    These  math  and  reading  scores  were  obtained  as  a 
measure  of  ability  in  order  to  observe  whether  better  choices 
(leading  to  better  success)  would  be  made  by  students  of  higher 
ability.     Figure  1  shows  the  frequency  distribution  of  the  math 
and  reading  scores. 

Figure  1:  Frequency  Distribution  of  Math  and  Reading  CAT  Scores 


528 


Drill  and  Practice  7 


Materials 

The  tedchers  of  these  students  were  consulted  to  detennine 
which  verbal  information  would  be  used  for  the  study.     This  was 
done  so  that  the  drill  might  enhance  regular  classroom  procedures 
rather  than  interfere  with  them,  and  so  that  the  material  used 
would  be  a  part  of  the  students'  program  of  study.    The  resulting 
drill  involves  the  naming  of  the  capitals  of  the  50  states  of  the 
United  States. 

A  computer-based  drill  program  was  developed  for  use  with 
this  verbal  information.    The  findings  of  research,  as  described 
above,  were  utilized  in  the  designing  of  the  drill  in  the 
following  ways: 

1)  Feedback  includes  discrimination  feedback;  i.e.  when 
a  student  answers  incorrectly  with  another  response 
from  the  list,  both  the  correct  ansu^r  and  the 
stimulus  for  which  their  response  should  be  given  is 
indicated.     If  the  response  is  not  in  the  list,  only 
the  correct  answer  is  given.  After  correct  answers, 

a  counter  is  incremented  and  the  next  item  appears. 

2)  Once  an  item  is  missed,  it  is  reviewed  three  times 
with  the  following  schedule:     after  2  intervening 
items,  after  3  more  intervening  items,  and  then 
after  5  more  intervening  items.     If  it  iz  missed 
again,  this  pattern  is  begun  over.     If  two  iteais  are 
cued  to  the  same  position,  the  item  most  recently 
incorrect  is  placed  first. 

3)  If  an  item  is  answered  correctly  the  first  time,  or 
if  it  has  passed  through  a  review  schedule  with  no 
further  errors,  it  is  moved  to  the  review  pool.  In 
this  way  the  size  of  the  working  pool  is  controlled. 

4)  The  drill  session  can  be  ended  at  any  time.  This 
allows  the  student  to  decide  the  amount  that  he/she 
scudies  at  one  time.    This  is  a  second  way  of 
controlling  the  size  of  the  working  pool. 

5)  Records  are  kept  of  each  practice  session. 
Therefore,  if  a  student  stops  before  completing  the 
entire  list  of  items,  at  the  next  session  the 
program  continues  where  it  left  off. 

The  drill  was  reviewed  by  four  persons  (one  professor,  two 
adults,  and  a  teacher  of  the  sample  group)  who  suggested 
editorial  changes  which  w^re  incorporated  into  the  prograin  before 
it  was  used  by  the  studen^.s. 

Pretest  an<?  Posttest 

Tiie  piatest  and  posttest  consisted  of  a  random  listing  of 
the  50  states  for  which  the  students  were  asked  to  give  the 
capital.    Both  tests  were  paper  and  pencil  tests,  as  a  sufficient 
number  of  computers  were  not  available  for  administration  to  a 
large  group. 

Procedure 

At  the  beginning  of  the  first  week,  the  material  was 
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assigned  and  the  written  prc^test  was  given.    The  computer-based 
uriXl  was  d6*ucnstrated  to  the  students  and  the  othsr  details 
involved  in  using  it  (location,  procedure,  etc.)  were  explained. 
The  students  were  given  two  weeks  for  the  learning  of  the 
matetial,  at  the  end  of  which  the  written  posttest  was 
administered.     The  choice  of  when  end  how  often  to  practice  was 
left  completely  to  the  student*    The  computer  recorded  each 
practice  session  and  its  results.    Each  student  had  a  folder  on 
file  containing  a  diskette  with  the  drill  program,  and  a  paper 
which  served  as  a  backup  to  the  information  recorded  by  the 
computer.     On  the  paper,  the  student  recorded  the  results  (number 
tried,  number  correct,  number  remaining)  on  blanks  arranged 
according  to  the  date  of  the  practice. 

Results 

Data  Collected 

Pretest  scores  ranged  from  0  to  4  3  (li=  55,  JJ=14.47,  SD=12.8) 
out  of  a  maximum  of  50.     For  an  item  to  be  considered  correct,  it 
had  to  be  spelled  correctly.     This  was  consistent  with  the 
computer  drill,  in  which  an  answer  was  not  considered  correct 
with  improper  spelling. 

During  the  two-week  period,  the  length  and  number  of 
practices  were  affected  by  computer  availability,  absenteeism  and 
classroom  schedule.     Student  use  of  the  drill  thus  ranged  from  0 
to  17  sessions,  with  a  median  of  2  sessions  (M=  62,  Jl=3.13, 
SD=2.9).     Length  of  practices  also  varied  widely,  from  an 
individual  student  average  of  5.5  items  per  session  to  an 
individual  student  average  of  148  items  per  session  fMdn=3  3 . 5) . 

At  T:he  end  of  the  practice  period,  posttest  scores  ranged 
from  1  to  50  (N=61,  M=27.54,  SD=18.15).     The  resulting  difference 
scores  (posttest  minus  precest)  ranged  from  --5  to  4  3  with  a  mean 
of  12.4  and  a  standard  deviation  of  12.28. 

Figure  2  shows  the  frequency  graphs  of  the  above  variables. 
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Fifty-one  of  the  students  (82%)  took  both  the  pretest  and 
posttest  and  used  the  drill  during  the  2-week  period.     All  of  the 
remaining  statistics  refer  to  this  group. 

New  Variables  Computed 

Because  the  length  of  practices  varied  widely,  a  variable 
was  created  to  indicated  number  of  items  tried  as  another  way  of 
quantifying  amount  of  practice.    This  vari=^ble,  which  counts  an 
item  each  time  it  is  encountered,  ranges  from  7  to  436  items 
tried  (M=122.43,  SD=112.4). 

A  third  variable  used  to  quantify  amount  of  practice  was  the 
number  of  times  a  student  completed  th^    ^ntire  drill.     It  ranges 
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from  0  to  6  with  a  mean  of  1.19  (Mdn=>54)  and  a  standard 
deviarion  of  1.45, 

To  quantify  the  student's  success  in  learning  unknown  items, 
the  difference  between  the  posttest  and  pretest  scores  was 
divided  by  the  number  each  student  needed  to  learn  (50  minus  the 
pretest  score) ,  creating  a  percentage  of  "success"  in  learning 
unknown  items.     It  ranged  from  -•12  to  1.00  (M=^»43,  Sj5=*39). 

A  variable  was  created  to  compare  practice  with  success.  It 
was  obtained  by  dividing  the  number  of  items  tried  by  the  number 
of  items  learned.    Thus,  it  indicates  the  nvimber  of  items  tried 
per  each  item  learned.     It  ranges  from  0  to  148,  with  a  mean  of 
13.95  and  a  standard  deviation  of  21.9. 

A  final  variable  was  created  indicating  when  the  student 
practiced.    The  coding  was  E  (early)  for  those  who  practiced  only 
the  first  week,  L  (late)  for  those  who  practiced  only  the  second 
week,  and  B  (both)  for  those  who  practiced  both  weeks. 

Statistical  Procedures  Used 

The  frequency  graphs  of  the  math  and  readin^  scores  indicate 
that  the  ability  level  of  the  sample  group  is  no  normally 
distributed  (see  Figure  1)  .    There  are  various  Ot>inions  as  to 
which  tests  should  be  used  with  such  samples.     For  this  reason, 
both  parametric  and  non-parametric  tests  were  used  for  many  of 
the  statistical  procedures.     In  all  cases,  basic  conclusions  were 
the  same,  with  differences  being  found  only  in  the  strength  of 
the  results.     For  example,  regression  analysis  on  the  rank  scores 
resulted  in  more  conservative  figures  than  regression  on  the  raw 
scores.    The  non-parametric  values  are  reported  in  this  paper. 

Comparison  of  Two  Groups 

22  students  learned  60%  or  more  of  their  unknown  items,  and 
29  learned  38%  or  less.    The  distribution  of  AMOUNT  LEARNED  is 
shown  in  Figure  3. 

Figure  3;  Frequency  Distribution  of  Amount  Learned 


Since  the  goal  for  the  students  was  to  learn  all  the  states  and 
capitals,  the  22  students  who  learned  at  least  60%  of  their  goal 
were  considered  "successful"  (S) ,  and  the  other  29  were 
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considered  "unsuccessful"  (U) .  These  groups  were  then  compared  in 
order  to  observe  what  may  have  determined  their  level  of  success. 

The  Wilcoxon  rank-sum  test  was  used  to  identify  significant 
differences  in  means  between  the  S  and  U  groups.     As  shown  in 
Table  1,  the  means  of  several  variables  varied  signif icant-ly 
between  the  two  groups. 


Table  1:  ComDarison  of  Means  of  S  and 

u  arouDS 

Usina 

the  wilcoxon 

Rank-Sum  Test 

MEAN 

p-value 

S 

U 

math 

82.08 

55.80 

.  0002 

reading 

76.70 

60.  54 

.  0170 

pretest  score 

22.  09 

9.56 

.0001 

number  of  sessions  of  practice 

4.46 

1.75 

.  0001 

number  of  times  through  the  drill 

2.28 

.32 

.  0001 

number  of  items  tried 

207.32 

52.59 

.  0001 

posttest  score 

45.96 

14. 16 

.  0001 

difference  (posttest  minus  pretest) 

23.86 

4.53 

.  0001 

These  results  indicate  that  the  successful  gioup  practiced  more 
times,  completed  the  drill  more  times,  and  tried  more  items  than 
the  unsuccessful  group. 

One  variable  did  not  differ  significantly  between  groups, 
and  that  is  the  number  of  items  tried  per  each  item  learned 
(E=.7973).     The  mean  of  this  variable  for  the  combined  groups  was 
13.95.     This  suggests  that  it  may  be  possible  to  quantify  the 
amount  of  practice  needed  in  order  to  learn  an  item. 

Agreement  with  Previous  Research 

Although  no  control  was  made  for  effects  of  the  pretest 
results  on  student  choices,  data  from  this  study  appears  to  agree 
with  previous  research  regarding  the  time  of  practice,  as 
discussed  on  page  4.     The  students  who  practiced  both  weeks  (both 
early  and  late)  numbered  23,  and  learned  an  average  of  65%  of 
unknown  items.     The  students  who  practiced  late  (N=14)   learned  an 
average  of  36%,  while  those  who  practiced  early  (N~12)  learned  an 
average  of  19%.     None  of  the  successful  students  (S)  practiced 
only  during  Monday-Wednesday  of  the  first  week.     Likewise,  none 
of  the  students  with  a  success  rate  less  than  10%  (N=ll) 
practiced  both  weeks  or  at  the  end  of  the  second  week. 

Predictors  of  Amount  Learned 

Spearman  rank  order  correlation  coefficients  were  run  on 
pairs  of  variables  to  note  any  strong  relationships  between  them. 
These  results  are  reported  in  Table  2. 
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Table  2;  Spearman  Rank  Order  Correlation  Coefficients 
Between  Variables 


No. Times 
Through 


No.  of 
Items 
Tried 


Percent 
Learned 


Variables 


Math 

Reading 

Pretest 

No.  Times  Through  Drill 


DjaJLi 
.36 
.34 
.47 


.33 
.28 
.34 

s31 


.44 
.31 
.53 
.76 


Speamiin  Rwik  Orckr  Correlation  Coefficients  of  ,37  or  more  have  a  p-value  of  .01  or  less. 
Those  of  .28  or  more  have  a  p*vaiue  of  .05  or  less. 


The  Ji-values  suggest  that  success  correlates  more  x^ith  the 
number  of  times  through  the  drill  than  with  number  of  items 
tried.    The  strong  relationship  between  number  of  times  through 
the  drill  and  nximber  of  items  tried  results  from  the  fact  that 
they  both  measure  amount  of  practice •    The  ^-values  in  the  first 
two  rows  of  the  table  suggest  that  ability  level  (as  provided 
through  the  math/reading  CAT  scores)  was  not  strongly  correlated 
with  the  choices  made  by  the  students. 

Regression  analysis  was  performed  with  the  amount  learned 
(percentage  of  success)  as  the  dependent  variable,  and  with 
pretest  score  and  the  number  of  times  through  drill  (or  number  of 
items  tried)  as  independent  variables.    The  number  of  times 
through  the  drill  contributed  58%  of  the  variance  and  the  pretest 
added  another  4%  (e=.03).    The  other  combination  (pretest,  number 
of  items  tried),  yielded  an       of  59%  (e=.001). 

A  second  analysis  was  done  with  the  same  variables,  but 
first  controlling  for  the  effect  of  the  pretest  score.  The 
pretest  score  explains  28%  of  the  amount  learned  (p^.OOOl). 
After  the  effect  of  the  pretest  score  is  removed,  the  number  of 
items  tried  explained  38%  of  the  amount  learned,  while  the  number 
of  times  through  the  drill  explained  36.5%  of  the  amount  learned 
(E=.0001  for  each). 

Item  Analysis 

The  computer  recorded  the  status  of  each  item  at  the  end  of 
a  practice  session.     In  this  context,  an  item  was  considered  as 
practiced  once  either  when  it  was  answered  correctly  and  thus 
encountered  once,  or  when  it  was  missed  and  therefore  repeated  3 
times.    This  information  was  studied  item-by-item  in  order  to 
observe  at  what  point  an  item  appeared  to  be  learned  by  the 
student.     Since  30  of  the  51  students  did  not  complete  the  drill 
even  once,  many  of  the  items  were  not  practiced  at  all. 

With  51  students  potentially  practicing  50  different  items, 
there  was  a  total  of  2550  items  to  consider.     Forty-two  percent 
(1065  items)  were  not  practiced  at  all.     Of  the  remaining  1485 
items,  58%  (861  items)  were  practiced  once,  15%  (218  items)  were 
practiced  twice,  17%  (256  items)  were  practiced  3  times,  7%  (100 
items)  were  practiced  5  times,  and  3%  (50  itexns)  were  practiced  6 
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times. 

The  percentage  of  items  correct  on  the  posttest  after  the 
various  number  of  times  of  practice  was  calculated.  Percentages 
were  calculated  on  all  items ,  as  well  as  on  those  which  were  not 
known  at  the  time  of  the  pretest.    These  results  are  shown  in 
Figure  4. 

Figure  4;  Percent  of  Items  Correct  After  Practice 


From  this  graph,  it  is  possible  to  see  that  once  an  item  was 
practiced  twice,  it  was  about  as  likely  to  be  answered  correctly 
on  the  posttest  as  if  it  were  practiced  more  than  twice  (whether 
it  was  known  at  the  pretest  or  not) .     This  suggests  that  two 
times  through  the  drill  may  be  sufficient  for  learning  most  of 
the  items. 

Using  average  figures  from  the  data  on  each  item  practiced, 
it  was  possible  to  approximate  the  number  of  items  tried  per  each 
item  learned  while  completing  the  drill  twice,  as  follows:  The 
average  "student"  (out  of  50  states  and  capitals) , 
had  19  correct  on  the  pretest 
knew  29.5  after  the  first  practice  (based  upon 

the  data  on  items  practiced  only  once) 
had  37  correct  on  the  posttest, 
thus  learning  18  items. 

During  the  first  practice,  this  "student"  would  have  tried 
the  19  problems  once  each,  and  tried  31  problems  4  times  each, 
for  a  total  of  143  items  encountered.    At  this  point,  he/she  knew 
29.5  items.     During  the  second  practice,  29.5  items  were  tried 
once  and  20.5  items  were  tried  four  times,  for  a  total  of  111.5 
items . 
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After  two  practices,  then,  the  average    student"  had  tried 
254.5  items.     Since  he/she  learned  18  items  between  the  pretest 
and  posttest;  he/she  tried    254.5/18    or  14.14     items  for  each 
item  learned.    This  figure  is  comparable  to  the  actual  mean  of 
13.95,  as  calculated  from  the  student  data.    This  fact  suggests 
that  both  the  item-by- item  analysis  and  the  variabxe  "number  of 
items  tried  per  each  item  learned"  are  indicating  that  practicing 
an  item  two  times  with  the  procedures  used  was  sufficient  for 
learning  most  of  the  drill  material. 

Discussion 

The  purpose  of  phase  1  of  this  study  was  to  obtain  data  from 
which  to  propose  an  optimal  schedule  of  review  for  the  learning 
of  verbal  information.    The  following  are  the  conclusions  drawn. 

The  ability  level  of  the  students,  as  indicated  from  their 
CAT  scores,  did  not  seem  to  strongly  affect  the  choice,  and  thus 
effectiveness,  of  their  practice  schedule.   (The  highest  £-value 
was  .36  between  the  math  score  and  number  of  times  through  the 
drill,  as  seen  in  Table  2.)     The  math  and  reading  scores  were 
therefore  not  used  for  further  analyses. 

The  number  of  times  through  the  drill  was  the  strongest 
factor  for  predicting  the  success  of  the  student.     This  was 
demonstrated  both  in  the  correlation  and  the  regression  analyses, 
as  well  as  in  the  item-by-item  analysis.    The  item-by-item ^ 
analysis  suggests  +-hat  two  times  through  the  drill  is  sufficient. 
Since  those  who  practicc»d  both  early  and  late  in  the  two-week 
period  were  more  successful,  the  two  practice  sessions  should  be 
scheduled  accordingly. 

The  conclusion  of  this  phase  of  the  study,  then,  is  that  for 
the  learning  of  verbal  information,  when  an  optimized  computer^ 
drill  (with  discrimination  training  and  an  increased  ratio  review 
of  3  positions  for  missed  items)  is  used  with  placement  of 
reviews  early  (near  the  time  of  original  learning)  and  late  (near 
the  retention  measure) ,  then  the-  two  practice  sessions  are 
sufficient.     In  phase  2,  these  conditions  will  be  tested  in 
comparison  to  practice  strategies  in  which  the  review  of  missed 
items  and/or  the  scheduling  of  practice  sessions  are  altered. 
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ABSTRACT 


Computer-assisted  instruction  (CAI)  first  became  a  major 
area  of  interest  during  the  1960s,  partially  because  equipment 
and  software  costs  had  moderated  by  that  time,  and  partially 
because  federal  funds  became  available  to  support  some  of  these 
applications  in  certain  settings*  Interest  in  CAT  has 
continued  to  escalate  as  microcomputers  have  become  more 
available  and  as  more  software  for  these  applications  has  also 
been  developed* 

With  the  advent  of  microcomputers  (such  as  Apple  II,  IBM 
PC,  TRS-80,  and  Commodore  PET)  research  in  K--12  and  colleges 
greatly  increased  since  1975 •  These  studies  were  designed  to 
determine  the  general  effectiveness  of  CAI  to  the  best  method 
of  textbook  or  other  specific  characteristics  of  CAI»  These 
studies  were  conducted  in  three  main  settings:  computer 
laboratories,  remedial  instruction,  and  non-remedial 
instruction* 

This  paper  reviews  empirical  studies  of  the  efficacy  of 
CAI  use  in  various  settings,  with  particular  attention  to  the 
use  of  CAI  to  teach  mathematics  skills  and  concepts.  Studies 
using  various  ages  of  students  and  subjects  are  considered,  and 
are  applications  in  which  CAI  was  used  as  the  primary 
instructional  vehicle  as  against  studies  in  which  CAI  was  used 
as  a  supplement  to  conventional  instructional  intervention* 
These  studies  are  used  to  discuss  design  flaws  in  CAI  research 
studies*  Recommendations  are  suggested  for  improving  CAI  study 
designs*  Alleviating  the  design  flaws  might  actually  show 
differences  in  achievement  not  yet  seen  but  in  a  few  studies* 

In  general,  CAI  has  been  found  to  be  at  least  as  effective 
as  conventional  instructional  methods,  but  superiority  of  CAI 
has  been  demonstrated  in  only  a  limited  number  of  studies* 
Failure  to  find  statistically  or  educationally  significant 
results  favoring  CAI  may,  in  some  cases,  be  an  artifact  of 
small  sample  size  or  other  design  flaws*  CAI  appears  to  be 
most  useful  when  software  includes  diagnostic  branching  that 
attempts  to  detect  the  conceptual  difficulties  that  students 
experience  when  confronting  certain  problems,  and  then  presents 
instructional  activities  that  focus  on  individual  student 
needs  * 
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can  CM  be  more  Effective  for  Teaching  Jenny  than  Traditional 
Instruction  -  Why  have  Studies  been  Inconclusive? 


The  1960s  was  the  initial  introduction  of  computers  into 
education.  Pioneered  by  IBM  and  Philco'-Ford,  computer-assisted 
instruction  (CAI)  was  developed  and  brought  computers  closer 
to  being  in  the  classroom.  In  the  late  19703,  inexpensive 
computers,  such  as  the  Apple  II,  *rRS-80  model  I,  and  Commodore 
PET,  were  available  for  K-12  schools.  This  began  a  trend 
toward  deveolping  vast  numbers  of  CAI  programs  and  CAI 
research • 

As  software  has  developed,  a  reorganization  of  thinking 
of  what  a  computer  can  do  has  emerged.  Gellespie  (1981) 
discussed  the  computer  being  considered  a  "tutor"  that  could 
provide  endless  practice,  present  alternative  paths,  provide 
self -pacing,  and  ask  many  questions  in  a  variety  of  formats. 
The  computer  also  has  other  advantages,  such  as  presenting  the 
same  material  to  many  students  simultaneously  while  serving  the 
needs  of  the  individuals. 

Since  about  1962,  researchers  have  explored  the 
effectiveness  of  educational  software  in  the  schools  under 
various  conditions.  Zinn  (1965),  Wold  and  Hunter  (1984),  and 
James  (1986)  have  argued  that  CAI  is  effective  for  instructing 
students.  But,  many  studies  that  these  authors  reviewed  to 
draw  their  conclusions  had  design  flaws.  In  addition,  meta- 
analyses have  been  performed  which  contend  that  CAI  is  at  least 
not  harmful  and  probably  effective  (Burns  &  Bozman,  1981; 
Hartley,  1978;  Hasselbring,  1986;  Kulik,  Bangert,  &  Williams, 
1983)  . 

Computers  have  become  part  of  the  school  environment. 
This  is  evidenced  by  the  tripling  of  the  number  of  computers 
used  for  instructional  purposes  in  K-12  schools  during  the 
first  half  of  the  1980s.  In  light  of  the  expanding 
availability  of  computers,  several  questions  have  arisen. 
First,  is  CAI  a  viable  method  to  effectively  instruct 
independently  of  other  instruction?  And  secondly,  why  have 
previous  studies  been  inconclusive  about  the  effectiveness  of 
CAI? 


Viability  of  CAI  to  Instruct 

In  the  past  few  years,  there  have  been  many  educators 
arguing  that  CAI  is  effective  for  instructing  students  (James, 
1986;  Wold  &  Hunter,  1984;  Zinn,  1965).    These  authors  describe 
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the  types  of  instructional  programs,  the  development  of  the 
software,  and  even  how  to  use  the  software*  What  is  lacking 
is  information  describing  the  successful  implementation  of  the 
software  with  student  achievement  improved*  Most  articles 
relate  to  the  viability  of  CAI  as  a  supplement  to  classroom 
instruction  or  independent  of  classroom  instruction.  But  the 
fundamental  question  is  can  CAI  aid  student  le-^rning?  The 
follow-up  questions  include:  whether  some  program  are 
effective  and  othere  are  not;  and  how  should  CAI  and  computers 
be  used  within  the  curriculum? 


Computer  Laboratories 

Several  studies  have  explored  the  effect  of  computer  labs 
in  which  students  "experiment  with  concepts  or  applications  or 
explore  repeated  applications  of  course  content'*  (Sterrett  and 
Karian  (1978,  p*  114)*  Sterrett  and  Karian  (1978)  used  a 
statistical  laboratory  in  an  introductory  non-calculus 
statistics  course.  The  35  students  in  two  statistics  classes 
were  given  accounts  and  data  sets  to  explore  the  concept  of 
statistical  significance.  Sterrett  and  Karian  (1978)  found 
that  the  use  of  this  lab  "enJianced  the  students'  ability  to 
understand  the  nature  of  experimental  results  and  statistical 
reasioning"  (p.  1116)  .  But,  statistically  significant  nor 
educationally  significant  results  were  found. 

Lang  (1973)  employed  CAI  as  a  supplement  to  introductory 
calculus  course  instruction.  The  CAI  supplemented  the  class 
work  of  19  students,  with  additional  homework  assignments 
intended  to  reinforce  several  topics  taught  in  class.  The  15 
non-CAI  students  received  similar  homework  assignments 
concerning  the  same  topics,  but  without  the  use  of  the 
computer.  Lang  (197  3)  found  that  concept  attainment  was 
enhanced  with  the  use  of  CAI.  Lang  (1973,  p.  5662A)  concluded 
that  CAI  has  the  potential  to  be  "an  effective  means  of 
improving  the  students'  understanding  of  the  basic  concepts  of 
calculus".  No  statistical  significance  was  found  for 
achievement  even  though  that  broad  conclusion  was  made. 

In  1975,  Calswell  and  Polley  created  a  computer  lab  which 
presented  students  with  practice  problems  with  randomly 
generated  problem  parameters.  The  students  solved  the 
problemss  and  entered  the  solution  into  the  computer.  The 
computer  would  diagnose  the  solution,  if  it  was  incorrect,  and 
match  an  error  to  the  answer.  Caldwell  and  Polley  (1975)  found 
that  this  diagnosis  was  helpful  to  some  students,  but  did  not 
eliminate  the  need  for  other  types  of  instructional  aids. 
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These  three  studies  illustrate  that  computer  can,  with  at 
least  moderate  success,  supplement  traditional  instruction* 
But,  from  these  studies,  computers  seem  an  improbable  solution 
for  eliminating  the  need  for  teachers  or  for  other 
instructional  aids. 


Remedial  Instruction 

CAI  has  also  been  employed  for  remedial  instruction.  CAI 
has  been  used  both  as  a  supplement  to  traditional  instruction 
as  well  as  the  sole  means  of  instruction  for  remediation. 
During  the  1970s,  many  educators  felt  this  was  the  dominant 
role  that  CAI  should  take. 

Sherman  (1975)  conducted  a  project  evaluating  the  use  of 
CAI  to  remediate  the  multiplication  and  division  skills  of 
college  students.  Sherman  selected  all  students  scoring  zero 
percent  on  a  diagnostic  test.  The  35  experimental  group 
students  received  CAI  while  the  82  control  group  students 
received  "traditional"  lecture  on  the  same  material.  Sherman 
(1975)  found  that  CAI  successfully  facilitated  student 
achievement  on  all  insstructional  objectives. 

In  1984,  Glidden,  Harrald,  Hodges,  and  Kunec  supported  the 
conclusion  made  by  Sherman  with  Coast  Guard  entrants.  These 
researchers  reported  statistically  significant  treatment 
effects  as  measured  by  the  Armed  Forces  Vocations  Aptitude 
Battery . 

Imboden  (1985)  studied  the  effects  of  computer  enhanced 
instruction  on  low  achieving  college  students  learning  the 
concept  of  percent.  Thirty-eight  subjects  in  the  freshment 
through  junior  year  were  assigned  to  either  the  experiir^ental 
group,  computer  enhanced  instruction,  or  the  control  group, 
lecture  method,  with  both  groups  covering  the  material  to 
essentially  the  same  degree.  Imboden  (1985)  found  that  the 
computer  enhanced  instruction  appeared  not  to  be  different  from 
the  lecture  method. 

These  studies  highlight  that  CAI  is  a  viable  means  of 
presenting  material  for  remedial  instruction.  All  of  the 
studies  discussed  used  the  CAI  for  sole  instruction  but  also 
had  a  narrow  focus.  No  studies  were  found  that  used  CAI  for 
remediation  and  used  it  as  a  supplement  to  "traditional" 
instruction.    This  indicates  a  gap  in  the  research. 
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Non-Remedial  Instruction 

The  use  of  CAI  as  a  means  of  non-remedial  instruction  has 
been  the  foucs  of  various  studies  involving  students  from 
elementary  to  post  graduate  educational  levels.  Research 
ranges  from  the  investigation  of  using  CAI  to  teach  specific 
topics  to  using  CAI  to  teach  the  entire  course.  Two  recerc 
studies  illustrate  the  use  of  CAI  to  teach  an  entire  course. 

Enochs,  Handley,  and  Wollenberg  (1986)  used  a  stratified 
random  sample  dividing  50  U.S.  Navey  Yoeman  "A"  students  into 
two  groups:  "those  taught  traditionally  in  self -paced  mode  and 
those  taught  via  CAI"  (p.  136).  Enochs  et  al.  (1986)  found 
that  there  were  statistically  significant  differences  in 
achievement  gains  across  croups  which  favored  the  CAI  group, 

Ozarowski  (1973)  investigated  the  use  of  CAI  in  college 
statistics  classes.  Ozarowski  (1973)  assigned  23  subjects  to 
each  group,  experimental  and  control,  by  matching.  The 
students  covered  the  same  material  normally  undertaken  in  a 
basic  statistics  course.  The  researcher  found  that  the  time 
taken  to  complete  the  course  was  statistically  significant  less 
when  the  computer  method  was  used  rather  than  the  conventional 
classroom  method  of  instruction.  Learning  gains  were  also 
statistically  significant  in  favor  of  the  students  being 
instructed  by  the  computer  (Ozarowski,  1973).  These  results 
are  even  more  noteworthy  given  the  relatively  small  sample 
size. 

The  studies  by  Encohs  et  al.  (1986),  Imboden  (1985),  and 
Ozarwski  (1973)  indicate  that  CAI  is  at  least  as  effective  as 
"traditional"  instruction  when  CAI  is  used  to  teach  an  entire 
course.  There  is  also  evidence  that  CAI  is  better  than 
"traditional"  instruction  in  the  length  of  time  students  need 
to  learn  material  and  the  achievement  level. 

As  a  note,  there  are  numerous  CAI  studies  conduct  with  one 
to  five  topics.  Thse  studies  show  nominal  achievement  gains 
in  favor  of  CAI,  but  never  statistically  significant 
differences.  The  lack  of  statistical  significance  are  due  to 
the  small  sample  sizes,  but  even  a  greater  problem  is  the 
shortness  of  the  study.  Thus,  there  may  be  more  effectiveness 
in  CAI  than  these  studies  can  ever  determine  because  of  design 
flaws. 
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Reviews  of  educational  software  effectiveness  (Chambers 
&  Sprecher,  1980;  Colorado,  1988;  Schmitt,  1987;  Schmitt,  1989) 
studies  have  noted  four  major  research  design  weaknesses: 
small  sample  sizes,  lack  of  identified  criteria  for  determining 
"quality"  software,  inappropriately  used  statistics,  and 
inadequate  time  allocated  to  conducting  the  study.  Because  the 
actual  studies  have  design  flaws,  these  research  design 
weaknesses  will  be  reflected  in  the  results  of  the  studies  and 
even  compounded  should  meta-analyses,  using  these  studies,  be 
performed  since  meta-analyses  are  analyses  of  an  analy^sis  of 
data* 

This  section  discusses  these  four  major  weak: .esses  in  many 
CAI  studies.  Sixteen  studies  are  used  to  illustrate  the  design 
weaknesses  frequently  found  in  CAI  research  but  is  also  common 
in  other  educational  research  endevors. 


Selection  of  Research  Studies 

Several  sourses  of  educational  papers  were  surveyed,  ERIC, 
Dissertation  Abstracts  Internation,  and  educational  journal, 
from  1970  through  1987.  Sixteen  CAI  studies  in  mathematics 
published  during  this  17  year  period  were  randomly  selected  to 
be  used  to  illustrate  the  research  design  weaknesses  in  CAI 
studies. 

The  appenidx  has  a  table  lists  the  various  characteristics 
of  CAI  studies  in  mathematics.  This  tible  present  the  sample 
size,  the  length  of  the  study,  the  software  used  (commercial, 
public-domain,  self-made,  or  unknown),  the  statistical 
procedures  employed,  and  the  conclusions  (more  effective,  no 
difference,  less  effective)  with  traditional  instruction. 


Small  Sample  Sizes 

One  controlable  research  design  aspect  is  the  sample  size. 
The  second  level  questions  the  researcher  asks  are: 
Who  is  my  population? 
Who  will  be  in  my  sample? 

And,  how  many  people  will  be  in  my  sample? 
These  three  questions  usually  follow  the  definition  of  the 
research  problem  and  development  of  the  research  questions. 
Tables  and/or  formulas  can  be  found  in  any  number  of  social 
science  statistics  books  to  determine  the  sample  size  required 
to  have  sufficient  "power"  for  the  study. 
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CAI  researchers  like  many  other  educational  researchers 
neglect  sample  size.  The  mean  sample  size  in  these  16  CAI 
studies  reviewed  was  74.  This  is  the  sample  size  for  the 
entire  study,  not  just  a  level  of  a  way.  Difficulty  achieving 
statistical  significance  is  suprising  due  to  this  fact.  Most 
CAI  studies  do  not  have  sufficient  "power".  Not  only  are  these 
studies  not  statistically  different,  but  if  educational 
significance  is  explored,  through  effect  sizes,  many  of  these 
same  studies  do  not  indicate  educational  significance  in  the 
findings.  This  is  also  supported  by  conclusions  by  (Colorado, 
1988;  Schmitt,  1987). 

As  examples  of  small  sample  sizes,  Pachter  (1979)  used  a 
sample  of  29.  Other  studies  using  a  sample  size  in  the 
thirties  were  Sterrett  and  Karian  (1978),  Lang  (1973), 
Robichaud  (19>36),  Imboden  (1986),  and  Wilson  (1986)  with 
samples  between  29  and  38.  Three  studies  did  have  large 
samples:  Goodson  (1975)  with  93,  Sherman  (1975)  with  117,  and 
Shapiro  (1977)  with  400  subjects. 

There  are  solutions  to  this  problem.  Instead  of  using  a 
single  school  and  one  grade  level,  the  researcher  could  use  two 
schools.  One  school  could  be  the  control  group  and  the  other 
could  be  the  experimental  group.  If  more  than  two  groups  are 
needed  expand  the  number  of  schools.  This  would  increase  the 
sample  two  fold  or  by  the  number  of  research  groups  needed  for 
the  study. 

A  second  solution  would  to  take  multiple  schools  and  have 
all  of  the  research  groups  microcosums  of  the  research  study » 
Instead  of  having  three  schools,  each  school  representing  a 
different  group,  have  each  school  have  the  three  groups  thus 
amounting  to  a  similar  sample  size  as  the  above  suggestion. 

A  final  suggestion  would  have  several  researchers  from 
different  school  districts  or  states  work  on  the  same  study. 
Each  researcher  would  have  all  the  groups  in  his  "sub-study". 
This  would  not  only  increase  the  sample  size  but  also  the 
generalizability  of  the  study. 

These  soultions  are  just  three  logically  thoughout 
solutions  to  the  problem  of  sample  size.  This  author  feels 
that  one  of  the  major  reasons  statistical  and/ or  educational 
significance  has  not  been  found  is  due  to  sample  size*  The 
lack  of  greater  effectiveness  over  traditional  instruction  has 
hindered  the  expanded  use  of  CAI  in  some  form  of  classroom 
instruction.  The  findings  of  these  past  studies  has  also 
hindered  the  further  investigation  of  CAI  effectiveness  similar 
to  the  past  and  in  other  aspects. 
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Duration  of  the  Study 

The  duration  of  the  study  is  usually  largely  under  the 
control  of  the  researcher.  Although,  the  availability  of 
students  and  the  liberties  schools  and  teachers  give 
researchers  may  be  constraining,  the  researcher  does  have  a 
reasonable  amount  of  control  over  how  long  the  study  will  take 
place* 

The  second  design  flaw  found  in  many  CAI  research  studies 
is  the  length  of  the  study.  Several  of  the  studies  (i.e., 
Imboden^  1986;  Shine,  1983;  Pachter,  1979;  Goodson,  1975)  only 
lasted  through  one  topic.  In  science  and  mathematics^  a  topic 
may  last  one  to  three  days.  This  limits  the  study  to  the 
length  of  time  the  single  topic  is  discussed.  Other  studies 
lasted  through  multiple  topics.  Finally,  there  were  some 
studies  that  lasted  a  semester  or  longer:  Sterret^c  and  Karian 
(1978),  Lang  (1973),  Crenshaw  (1983),  and  Ozarowski  (1973). 

The  length  of  time  is  largely  determined  by  the  number  of 
software  packages  are  used  in  the  study.  On  one  dimension 
there  are  the  topics  that  will  be  i  eluded  in  the  study  (i.e., 
solving  linear  equations,  long  vowels,  the  solar  system).  On 
another  dimension,  usually  neglected,  is  the  variety  software 
package  types  that  can  be  used  (i.e.,  drill  and  practice, 
tutorial,  simulation  and  demonstration,  games).  The  studies 
report  the  topics  and  usually  only  employ  one  type  of  software 
package,  A  more  accurate  deption  of  how  effective  CAI  is  on 
a  given  topic  might  be  to  use  several  differnt  software  types 
on  the  same  students  or  on  different  student  and  observe 
differences. 

Even  those  studies  which  lasted  a  semester,  the  students 
may  have  only  had  limited  use  of  the  computers.  Use  once  a 
week  or  every  other  week  could  possibly  not  show  differences 
as  much  as  using  the  computers  twice  or  three  times  a  week 
throughout  the  semester. 

This  is  not  to  say  that  the  general  duration,  a  semester 
or  even  multiple  topics  is  not  long  enouth,  but  this  author  is 
suggesting  that  actual  computer  usage  would  be  a  better 
indicator.  CAI  studies  have  varied  from  students  using  the 
computer  15  minutes  per  session  to  one  hour  per  session.  The 
duration  of  the  sessions  makes  a  dramatic  difference  when 
discussing  ten  15  minute  sessions  compared  to  ten  one  hour 
sessions. 
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Additionally,  if  the  sessions  are  distributed  over  various 
topicr  and  multiple  software  types  are  used,  there  is  more 
likely  to  be  statistical  and/or  educational  significance.  The 
length  of  the  study  most  def initionally  has  an  effect  on  either 
measure  of  significance.  And  obviously,  the  results  will  also 
be  more  stable  over  time  and  over  software  types.  This  may 
also  produce  more  generalizable  results. 


Softwjire  Quality 

The  lack  of  knowledge  of  the  software  quality  or  the  lack 
of  quality  software  used  is  considered  an  inexcusable  design 
flaw.  Educational  software  quality  ranges  from  abonable  to 
excellent  (Allen,  1934;  Bitter,  1984;  Burke,  1982;  Jensen, 
1985;  Kamil,  1984 ;  Komoski,  1987;  Lathrop  &  Goodson,  1983 ; 
Staples,  1985;  Taylor  &  Ely,  1987).  Staples  (1985)  has  stated 
that  there  is  a  tremendous  amount  of  variation  in  the  quality 
of  software,  but  the  largest  quantity  is  the  poorer  software. 
With  the  amount  of  poor  quality  software,  the  "educational 
merits  of  [software]  packages  need  to  be  questioned"  (Niman, 
1987,  p.  57)  . 

All  of  the  studies  reviewed  either  did  not  state  what 
software  was  used  or  defined  the  software  as  made  by  the 
researcher  (self-made).  There  was  no  discussion  in  any  study 
pretaining  to  the  quality  of  the  software.  This  is  disturbing 
since  one  major  criticisms  of  CAT  by  teachers  is  that  "there 
is  no  good  software  out  there"  (Balajthy,  1988,  p.  51). 

The  quality  of  the  software  is  the  backbone  of  all  CAI 
studies.  If  the  software  is  not  excellent  or  at  least  good, 
then  CAI  may  be  found  to  be  ineffective  or  at  least  as 
effective  as  "traditional"  instruction.  The  quality  or  lack 
there  of  in  the  research  software  is  compounded  by  the  previous 
two  flaws,  particularly  the  duration  of  the  study. 

The  major  concern  for  this  author  is  not  so  much  that  the 
researchers  developed  the  software  for  the  study,  but  that  the 
quality  of  the  software  was  not  documented.  This  is 
particularly  a  concern  in  those  papers  from  ERIC  since  there 
is  no  page  limitations. 

Suggestions  for  documenting  the  quality  of  software 
whether  it  may  be  commercial,  public-domain,  or  self-made. 
First,  if  the  software  being  used  in  the  study  was  commercially 
produced,  check  in  ERIC  or  subject  matter  journals  for  reviews 
of  the  software  (i.e.,  MicroSIFT,  Alberta,  Arithmetic  Teacher, 
Mathematics  Teacher,  English  Teacher,  Reading  Teacher,  Science 
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Teacher) .  Second,  if  the  software  was  not  commercially 
produced,  locate  a  software  evaluation  instrument  (i.e., 
MicroSIFT,  NCTM,  NCTE,  or  other  published  in  ERIC  or  various 
journals) .  Third,  provide  documentation  from  referencing  the 
review  found  or  a  summary  of  the  software  evaluation.  Finally, 
discuss  reason  for  using  particular  software  particularly  if 
the  software  was  self-made  and/or  received  a  poor  review  or 
evaluation. 

It  is  this  authors  opinnion  that  the  major  downfall  of 
past  CAI  research  studies  has  been  the  lack  of  quality 
software.  This  has  placed  both  CAI  research  and  the  use  of  CAI 
software  in  jepordy.  Without  credible  software  research 
studies  will  fail  to  accurately  determine  eff fectiveness  or 
ineffectiveness  of  CAI. 


Statistical  Procedures  Employed 

To  this  point,  most  studies  have  relied  on  univariate 
statistics.  The  predominant  statistical  procedures  employed 
have  been  analysis  of  variance  (ANOVA)  and  t-tests.  In  and  of 
themselves  there  is  nothing  wrong  with  these  statistical 
procedures,  but  they  have  been  misused  and  overused. 

The  three  statistics  that  were  used  by  all  of  the  reviewed 
studies  were  ANOVA,  t-test,  and  regression.  Given  the  data, 
the  general  type  of  statistics  used  was  appropriate.  One 
problem  is  that  in  several  studies,  multiple  ANOVAs  and 
multiple  t-tests  were  employed.    This  is  problematic. 

The  same  logic  used  by  textbooks  supporting  the  use  of 
ANOVA  tests  over  f  tests  can  be  made  for  using  multivariate 
statistics  over  multiple  ANOVAs.  If  the  alpha  is  set  at  .01, 
with  one  ANCVA  there  is  a  5  percent  chance  that  the  test  would 
be  declared  statistically  significant  (Asher,  1976) .  This  is 
termed  experimentwise  Type  I  error. 

When  more  than  one  ANOVA,  or  any  other  univariate  test, 
is  performed  on  the  same  dependent  variables  the  experimentwise 
Type  I  error  probability  is  increased  exponentially.  This 
error  can  range  from  the  set  alpha  level,  when  all  of  the 
dependent  variables  are  perfectly  correlated,  to  "1  -  (1  - 
alpha)  raised  to  the  Jc  power,  where  k_  is  the  number  of 
statistical  test  conducted"  (Thompson,  1986,  p»  6),  when  all 
of  the  dependent  variables  are  perfectly  uncorrelated.  The 
most  troubling  fact  is  that  except  at  the  two  extremes,  the 
actual  experimentwise  error  rate  cannot  be  readily  determined 
(Thompson,    1986) .      The  result   is  a  greater  probability  of 
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statistical  significance  (McMillan  &  Schumacher,  1984;  Sowell 
&  Casey,  1982) .  Thus,  instead  of  multiple  univariate 
statistics/  one  multivariate  statistic  should  be  employed. 

But  beyond  the  fact  that  the  probability  of  statistical 
significance  is  greater ^  multivariate  statistics  also  better 
represent  the  reality  of  the  classroom.  And^  the  classroom  is 
where  the  CAI  will  be  used.  Thompson  (1986)  argues  that  the 
statistical  procedures  should  be  as  complicated  as  the 
environment  in  with  the  data  was  collected.  The  classroom  is 
not  simple.  And,  there  is  not  one  dependent  variable  that  is 
involved.  Achievement,  interest ^  and  attitude  are  just  three 
aspect  that  are  involved  in  all  learing^  whether  or  not  the 
computer  is  used.  These  three  aspects ^  and  possibly  others ^ 
need  to  be  assess  together  and  not  independently. 
Additionally,  differet  aspects  of  achievement  should  not  be 
investigated  independently,  but  together  as  well. 

CAI  research,  like  much  of  educational  research ^  has 
neglected  the  need  and  usefulness  of  multivariate  statistics. 
Univariate  statstics  can  be  used.  And  there  are  appropriate 
situations.  But,  when  multiple  univariate  tests  need  to  be 
employed,  multivariate  tests  are  thJfen  most  appropriate. 

Finally,  educational  significance  or  effect  sizes  has  been 
neglected  throughout  CAI  research.  An  effect  size,  often  E  9 
is  used  to  determine  educational  significance.  One  of  the 
easiest  effect  sizes  to  calculate  is  (sum  of  square  explained 
divided  by  sum  of  squares  total)  .  Even  is  there  is  no 
statistical  significance,  especially  with  a  small  sample  size, 
there  may  be  educational  difference.  And  obviously,  the  higher 
the  percentage  of  difference  the  greater  the  difference. 


Design  Flaws  Affecting  Meta-Analyses 

Simply  put,  meta-analysis  is  the  "analysis  of  analyses" 
(Glass,  1976,  p.  3)  .  Basically,  meta-analysis  is  the  reviewing 
of  a  collection  of  studies  to  summarize  statistics  (Kulik  & 
Kulik,  1988)  .  Kulik  and  Kulik  (1988)  e:^:plain  that  meta- 
analysis is  not  based  on  statistical  significance  by  effect 
sizes  and  the  strength  of  relationships. 

Because  meta-analyses  rely  on  other  studies,  the  meta- 
analysis study  incorporates  any  problems  the  reviewed  studies 
might  have.  Thus,  CAI  meta-analyses  also  incorporate  the  flaws 
described  above.  Most  specifically,  length  of  the  study  and 
the  quality  of  the  software.  Due  to  the  use  of  effect  sizes, 
the  sample  size  does  not  influence  the  meta-analysis  as  much 
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as  if  statistical  significance  was  investigated,  since 
statistical  significance  is  largely  a  product  of  sample  size. 

The  flaws  of  the  studies  do  influence  the  results  of  meta-- 
analyses. This  compounds  the  problems  since  many  educators 
rely  on  the  meta-analyses  as  the  literature  for  the 
effectiveness  of  CAI  because  of  the  vast  numbers  of  CAI 
studies.  The  meta-analyses  can  provide  a  false  sense  of 
success  with  CAI  that  has  not  been  shown  to  necessarily  be 
present  in  CAI  studies. 


Summary 

There  are  design  flaws  with  past  CAI  effectiveness 
studies.  After  reading  a  few  studies  this  conclusion  seems 
obvious.  But,  all  is  not  lost.  If  researchers  heed  the 
recommendations  for  improving  CAI  studies  there  may  be  some 
hope. 

As  the  technology  of  computers  grows  and  the  demand  for 
computers  in  schools  from  within  the  education  profession  and 
from  parents  increases,  there  needs  to  be  accountability  that 
CAI  is  useful,  effective,  and  necessary.  Unless  there  are 
sound  research  studies  to  backup  the  use  of  CAI,  it  will  fall 
like  a  dead  duck. 

CAI  may  not  be  effective  as  the  sole  means  of  instruction, 
but  that  does  not  mean  it  is  not  useful  and  necessary,  the  jury 
is  still  out.  But,  CAI  does  seem  viable  for  remediation  and 
long  self  directed  courses.  The  CAI  researchers  need  to 
provide  the  fuel  to  spark  the  demand  for  quality  CAI  and 
appropriate  uses  of  the  software. 
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APPBMDIl 
TABLES  OF  STUDIES 

Table  1.    Mathematics  studies 


Author (s) 

Sample 
Size 

uurauion 
of  Study 

CAI 
Software 

Statistics 

Rockhill  (1971) 

53 

4  topics 

self 

T-tests 

Schoen  (1972) 

50 

Semester 

self 

ANOVA 

Lang  (1973) 

34 

Semester 

self 

ANOVA 

Ozarowski  (1973) 

46 

Semester 

self 

T-test , 
ANOVA 

C7Coason  \±zf/o) 

93 

1  topic 

sell 

1v^T/^T7A 
ANOVA 

Sherman  (1975) 

117 

Semester 

self 

ANOVAs 

Shapiro  (1S77) 

400 

Semester 

self 

ANOVA 

ouerreuu  «  ixanan  (xy/o) 

Semester 

self 

ANOVA 

Jrciu>llU6^    \        '  / 

1  topic 

self 

T-test 

bemesuer 

CCC 

T-tests 

Shine  (1983) 

41 

1  topic 

self 

ANCOVA 

Walker  &  Azumi  (1985) 

153 

5  months 

MATHl-6  T-tests, 
program      regres . 

Enochs  &  others  (1986) 

50 

Semester 

self 

ANOVA, 
Pearson 

Imboden  (1986) 

38 

1  topic 

self 

ANOVA 

Robichaud  (1986) 

32 

Semester 

self 

T-tests 

Wilson  (1986) 

38 

6  topics 

self 

T-tests 

mean  sample  size  79;  standard  deviation  92.3 

mean  sample  size  without  Shapiro  57;  standard  deviation  35.8 


Note.    CCC  is  Computer  Curricular  Corporation  System 
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This  work  explores  the  interplay  between  various  types  of  software  interactivity  and  a 
program's  motivating  elements.  It  is  our  hypothesis  that  children  bring  specifiable  expectations  to 
their  interactions  with  educational  software.  These  expectations,  to  a  large  extent,  will  determine 
which  features  of  a  given  program  will  be  motivating  to  a  child.  Additionally,  the  different  genres 
of  mteractivity  relied  upon  by  software  designers  will  elicit  different  expectations  from  r  ;hild  and 
may  resonate  or  conflict  willi  the  motivating  elements  built  into  the  program.  The  state  of  affairs 
for  the  designer  cf  educational  software  is  much  like  lliat  for  the  movie  director.  The  director  of  a 
certain  genre  must  include  the  elements  the  audience  comes  to  sec  (e.g.,  the  scarcy  scenes  of  a 
horror  movie)  and  not  fully  replace  or  confuse  them  with  elements  that  woric  for  oilier  genres  (e.g., 
slapstick).  Similarly,  the  software  designer  must  be  attentive  to  the  connections  between  the 
interactive  elements  upon  which  she  relies  and  thase  elements  which  are  appealing  to  a  child. 

The  research  reported  in  this  paper  represents  an  attempt  to  establish  the  validity  of  the 
above  position  prior  to  the  development  of  verification  oriented  research.  We  think  this  is 
pecessary  as  our  conceptual  approach  is  somewhat  different  than  prior  research  in  this  area. 
Previous  attempts  to  schematize,  or  categorize  software  have  often  been  done  according  to  adult 
conceptualizations  (e.g.,  Jonassen,  1985;  Taylor,  1980).  According  to  our  hypothesis,  it  is  the 
child's  perception  and  categorization  of  programs,  particularly  the  interactive  aspects,  which 
determine  how  we  should  think  about  designing  software  (cf.  Shapiro,  1987;  Winograd  &  Flores, 
1986;  Turkic,  10S4).  Accordingly,  our  first  research  question  is  whether  interactive  formats  have 
a  strong  enough  mfiuence  on  how  children  categorize  software  so  that  we  may  speak  of  genr^s  of 
inter  activity.  A  second  way  our  research  differs  from  previous  approaches  is  that  some  of  the  best 
works  on  the  motivational  aspects  of  computer  software  have  tended  to  ignore  differences  across 
software.  For  example,  Malone  (1981)  writes  of  the  role  of  challenge  in  creating  motivation. 
Challenge,  however,  is  not  what  motivates  us  to  use  a  particular  word  processing  program.  Rather 
we  choose  those  programs  which  conform  to  our  needs  as  writers.*  We  intend  to  show  that  the 
motivating  elements  of  software  vary  from  progrL.n  to  program  and  are  dependent  to  a  large 
degree  on  the  interactive  premise  of  the  software. 

To  begin  exploring  the  relationship  of  interactive  software  and  motivation  we  collected 
three  sets  of  data: 

1)  By  having  fifth  graders  sort  software  titles  into  piles  of  "like"  programs,  we  investigated 
how  children  categorize  educational  software. 

2)  We  gathered  infonnation  about  the  motivational  power  of  piOgrams  by  asking  the 
students  to  indicate  on  a  scale  of  one  tc  three  Iiow  much  they  like  to  use  each  program. 

3)  Through  group  interviews  we  identify  the  interactive  elements  of  the  favored  software 
genre  -  adventures. 

All  of  the  studies  vere  conducted  with  fifth  graders  enrolled  at  Royale  Elements  j 
School  in  Daricn,  Connecticut.  The  Darien  School  District  has  an  ample  computer  program  which 
is  laboratory  based.  Children  use  uie  computer  lab  during  free  time  and  regularly  as  part  of  class 
time.  The  software  selection  and  usage  is  closely  unked  with  the  curriculum.  There  are  enough 
computers  for  children  to  work  individually  if  they  choose.  And,  a  full  75%  of  the  children  had 
computers  (or  video  games)  in  their  household.  We  comment  wiiere  we  feel  that  this  usage  profile 
may  impact  the  generality  of  our  findings. 

Genres  of  interactivity 

If  children  think  *  software  in  terms  of  mteractivc  format,  then  it  is  reasonable  to  expect 
them  to  use  siuiilarities  in  interactive  format  across  programs  as  a  basis  for  categoriz^ition  (cf,  Chi, 
Feltovich  &  Glaser,  1981).      To  decide  if  a  cluster  of  i^rogmur  coheres  by  interactive 


*  In  fairness,  Malonc  (Malonc  &  Lcppcr  1984)  later  added  an  inicrpcrsunal  dimension  of  moiivauonal  vanablcs  which 
capture  the  interactive  appeal  of  a  larger  array  of  software. 
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equivalences,  without  a  crisp  definition  of  interactivity,  could  be  problematic.  However,  we 
expected  that  this  problem  of  identification  would  not  be  loo  great.  It  was  our  prediction  that 
studenis  would  create  and  label  a  few  categories  which  unquestio.  u)ly  suggested  interactivity. 

Fourteen  chilren  (balanced  for  gendov)  each  received  a  stack  of  S6  index  cards  which  had 
an  educational  software  title  typed  on  one  side.  Children  were  directed  to  "sort  the  cards  into  piles 
of  programs  that  are  alike."  No  further  instructions  were  given.  Students  were  left  to  interpret 
"alike"  and  decide  how  many  piles  to  create.  The  groupings  were  recorded  and  children  were 
asked  to  label  each  pile. 

A  separate  manipulation  removed  programs  which  a  child  had  not  actually  used  We 
only  consider  the  26  programs  that  were  known  by  at  least  half  of  the  students  in  both  the  sorting 
task  and  the  motivational  ranking  task  below.  The  sortings  were  subjected  to  a  cluster  analysis 
using  the  Rao-Russel  coefficient  of  similarity  and  a  complete  linkage  approach.  The  label  or 
labels  which  occur  most  frequently  over  all  the  programs  within  a  cluster  are  used  as  names  for 
the  resulting  cluster. 

The  analysis  yielded  seven  clusters  of  programs  which  could  be  differentiated  according 
to  the  labels  assigned  by  the  students.  We  give  the  label  and  our  interpretation  of  the  cluster  along 
with  the  programs  which  were  included.  The  order  of  the  programs  and  clusters  listed  below 
replicates  the  ordiual  positions  within  the  final  clustering.  For  example.  Bank  Street  Writer  is 
most  closely  linked  to  LogoWriter  and  most  distant  from  Path  Tactics. 

Writing  or  Language  Arts 
These  programs  are  typically  seen  as  used  for  \^  riting  or  typing  practice. 
Bank  Street  Writer,  LogoWriter,  Paws»  Writer  Rabbit. 

Print-out  Programs 

This  category  is  defined  by  the  program's  emphasis  on  the  goal  of  printing  something  out. 
Print  Shop,  Print  Shop  Companion,  Time  Liner. 

Puzzles 

One  student  calleu  .his  type  of  program  "mind  puzzles."  This  category  revolves  around  deductive 
style  reasoning. 

Gertrude  s  Puzzles,  Puzzles  &  Posters. 

States  or  Social  Studies 

Tlicsc  programs  all  involve  learning  about  the  50  states.  However,  the  titles  alone  may  define  the 
category. 

Dataquest:  Th^  Fifty  States,  Game  of  the  States,  States  and  Traits,  Coast  to  Coast. 

Adventure/Joumev/Mvsterv 
The  anchoring  attribute  of  this  category  appears  to  be  computer  simulation  of  the  physical  world. 
One  student  made  the  astute  distinction  between  mystery  and  adventur;  as  one  of  solving  vs. 
looking.  These  two  types  of  programs  are  strong  sub-clu-^ters  within  this  category. 
Odell  Lake,  Oregon  Trails,  Goodell  Diamond,  Treasure  Hunter. 


*  Wc  lake  inicractiviiy  lo  mean  somclhing  broader  ihhn  and  prior  to  a  sLnple  distinction  between  computer  and  "passive* 
media.  As  such,  interactivity  is  a  potentially  unbounded  concept  ranging  in  examples  from  playing  a  game  of  icnnis  to 
reflecting  on  one's  own  thoughts.  In  attempting  lo  define  interacuvity,  one  could  easily  flounder  in  debates  over  false 
problems:  Can  tnie  interactivity  only  take  place  between  two  humans?  Is  drill  and  practice  less  or  more  interactive  than 
inquiiy 'based  sdtwarc?  Fortunately,  our  ultimate  goal  as  researchers  is  not  to  come  up  with  a  ncccssaiy  and  sufficient 
dcfiniuon  of  interactivity.  Indeed,  if  there  arc  concepu  which  arc  meaningful  Uhrough  the  family  resemblances  of  their 
examples,  mteraciivity  surely  mwst  be  one.  Rather,  our  goal  is  to  organize  discourse  about  those  shared  and  differing 
attributes  which  characterize  the  many  situations     call  mteractive. 
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Putting  thinps  together 
Students  have  to  construct  something  and  test  it 
Clock  Works.  Woodcar  Rally. 


Math 

This  category  includes  many  of  the  programs  that  deal  explicitly  with  number.  Within  this  cluster 
there  is  a  tendency  for  drill  and  praciice  programs  to  be  a^^sociated  with  one  and  another. 
Multiplication  Puzzles,  Subtraction  Puzzles,  Math  Blaster,  Circus  Math,  Quotient  Quest,  The 
Market  Place,  Path  Tactics. 

There  are  two  strong  dimensions  used  to  develop  the  categorizations:  content  area  and 
interactive  model.  The  three  content  area  clusters,  language  arts,  social  studies,  and  mathematics 
should  not  be  very  surprising.  Given  schools'  strong  partitioning  of  knowledge  into  curricular 
domains,  one  would  expect  children  to  bring  their  categorization  of  school  experience  to  their 
categorization  of  software.  This  effect  is  probably  heightened  by  Darien's  use  of  the  computer  lab 
within  class  periods. 

Of  greater  interest  for  our  purposes  arc  the  other  four  gioupings,  especially  the  printer, 
adventure,  ond  building  programs.  From  this  small  sample  of  children  and  programs,  it  can  be 
seen  that  children  arc  quite  attentive  to  inieiactivity  in  their  conceptualization  of  software.  Each 
of  these  represents  an  identifiable  way  of  interacting  in  and  with  the  world:  using  tools, 
navigating,  searching,  and  creating.  One  might  easily  imagine  uiat  les??  interactive  but  more 
frequently  tauted  aspects  of  software,  such  as  graphics  quality  or  engagingness,  could  have  been 
used  to  equate  the  software.  Interactive  models  are  not  limited  to  action  in  the  phsyica!  world, 
although  this  is  the  currently  favored  genre  of  fifth  graders.  There  also  appears  to  be  a  sub-cluster 
within  the  math  grouping  in  which  the  student  competes  with  an  animated  agent  within  the 
computer  thus  modeling  interpresonal  interactivity.  Undoubtedly,  there  are  many  oth'^r  models  of 
interactivity  upon  which  computer  programs  do  or  will  rely. 

Motivation  and  types  of  software 

If  programs  arc  equally  motivating  but  rely  on  different  models  of  interactivity,  it  is 
likely  that  the  motivation  originates  from  different  sources.  To  investigate  the  relationship  of 
motivation  to  interactive  lOrmats,  we  operationalized  the  motivational  strength  of  a  program  lo 
mean  how  often  a  child  likes  to  use  the  program.  22  fifth  graders  (balanced  for  gender)  completed 
a  questionnaire  indicating  how  often  (always,  sometimes,  never,  unknown)  they  \^  ould  choose  to 
use  each  of  the  56  programs  in  their  free  time.  Below  is  the  ranking,  from  most  motivating  to 
least,  of  those  programs  that  were  known  by  at  least  half  of  the  students.  A  score  of  three  would 
represent  the  most  motivating  program. 


(2 

50) 

Oregon  Trail 

(1 

83) 

Coast  to  Coast 

(2 

50) 

Print  Shop 

(1 

82) 

Multiplication  Puzzles 

(2 

45) 

Odell  Lake 

(1 

81) 

Time  Liner 

(2 

32) 

Bank  Street  Writer 

(1 

80) 

Gertrude's  Puzzles 

(2 

21) 

Print  Shop  Companion 

(1 

79) 

Game  of  the  States 

(2 

13  ) 

Wood  Car  Rally 

(1 

72) 

Cloclc  Wor)cs 

(2 

10) 

Goodell  Diamond 

(1 

69) 

states  and  Traits 

(2 

09) 

LogoWr iter 

(1 

68) 

Paws 

(1 

95) 

The  Marlcet  Place 

(1 

65) 

Circus  Math 

(1 

94) 

Path  Tactics 

(1 

58) 

Treasure  Hunter 

(1 

94) 

Quotient  Quest 

(1 

55) 

Math  Blaster 

(1 

92) 

Dataquest:  The  Fifty  States 

(1 

46) 

Writer  Rabbit 

il 

92) 

Puzzles  and  Posters 
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AltiUidcs  towards  software  programs  appear  to  be  fairly  well  defined  across  the 
population  as  there  is  a  highly  significant  effect  for  program  type  (F  =  6.9).  There  is  a  significant 
effect  on  motivation  when  clusters  arc  treated  as  groups  within  an  anova  (F  =  7.75).  Adventures 
and  the  programs  which  generated  printer  output  were  significantly  prefered  to  the  oilier  clusters 
(Scheffe  test  at  p  <  .05)  but  not  to  one  and  another.  We  will  discuss  the  source  of  this  preference 
in  the  next  section.  Interestingly,  there  is  an  interaction  between  gender  and  program  type  (F  = 
1.7,  p  <  .03)  but  not  between  gender  and  cluster  type  (F  =  1.45,  p  >  .22).  This  suggests  that  it  is 
not  math  or  writing  per  se  to  which  girls  and  boys  object,  but  rather  specific,  and  perhaps 
prototypical,  implementations  of  the  category  of  software.  However,  the  sample  is  too  small  to  be 
confident  Li  a  failure  to  find  effects  or  to  begin  parceling  out  interaction  effects. 

Oregon  Trail  and  Print  Shop  are  representative  programs  from  the  highly  motivated 
clusters  of  adventure  and  print-out.  These  two  programs  are  extremely  different  in  the  sorts  of 
interactions,  goals,  and  sources  of  satisfactions  a  student  can  expect  Although  one  might  wish  to 
claim  that  the  same  sorts  of  design  elements  make  each  motivating  (e.g.,  an  appropriate  level  of 
complexity  or  challenge),  this  seems  to  be  the  wrong  level  of  analysis.  We  suggest  that  what 
makes  each  program  so  successful  is  the  fit  between  the  reasons  the  child  wants  to  use  the 
program  and  the  interactive  environment  it  creates.  A  prugram  which  includes  navigational  goals 
sho?jld  create  a  spatial  environment  Being  able  to  jump  from  menu  to  menu  in  Oregon  Trail  (as  is 
the  case  in  Print  Shop)  would  dcsiroy  the  interactive  premise  of  travel.  It  would  undermine  our 
expectations  of  interactions  within  a  spatial  environment  The  program  would  loose  much  of  the 
motivational  power  it  gains  by  relying  on  our  enjoyment  and  knowledge  of  navigating  in  the 
world. 

The  source  of  interactive  expectations. 

If  a  child  peiceives  a  program  as  an  adventure  game,  she  prefers  to  see  a  first-person 
surrogate  carry  out  her  actions  within  the  program's  environment.*  On  tlie  other  hand,  in  a  math, 
drill  and  practice  setting,  no  such  surrogate  is  expected  (perhaps  io  move  numbers  around  on  the 
screen),  nor  does  it  seem  to  provide  motivational  enhancement  beyond  the  temporary  effects  of 
novelty.  Why  might  this  be  the  case?  In  simplest  fashion,  much  of  what  is  enjoyable  in  software 
borrows  from  what  is  recognized  as  an  enjoyable  activity  in  the  real  world.  Children  have  a  prior 
model  of  interaction  which  includes  surrogates  (e.g.,  dolls)  for  fantasies.  On  the  other  hand,  the 
activity  and  feedback  involved  with  worksheets  -  animated  or  not  -  have  never  included  the  notion 
of  a  surrogate.  There  would  be  a  mismatch  between  their  expectations  of  the  worksheet  genre  of 
interactivity  and  its  delivery  with  fantasy  elements. 

Our  central  tenet  is  that  children's  motivation  towards  educational  software  can  be 
understood  to  a  large  extent  through  the  specifiable  expectations  children  bring  to  the  different 
genres  of  software.  These  genres  of  software  are  created  and  identified  on  the  basis  of  the 
interactive  models  which  .st  upon  the  child's  experiences  in  the  world.  We  maintain  this 
position  because  children  bring  previous  real-world  interactive  experiences  to  their  understanding 
of  the  various  formats  of  educational  software.  It  is  easy  to  see  how  they  analogize  or  equate 
software  to  various  real-worid  activities.  (E.g.,  This  program  is  like  going  on  a  trip;  this  one  is 
like  taking  a  test;  this  one  helps  me  make  somefhin'^.)  They  must  import  the  goals  of  interactions 
in  the  world  to  find  satisfaction  within  software.  Software  does  not  create  goals  and  lasting 
motivation  out  of  a  vacuum  -  the  computer  is  not  that  inherently  motivating.  Rather,  a  piece  of 
software  will  borrow  the  motivating  goal  stmcture  and  the  attendent  interactive  environment 
which  supports  a  particular  type  of  interaction  in  the  world.  The  two  most  highly  motivating 
clusters  of  programs,  adventures  and  printing  software  are  notable  in  their  -  lose  linkage  with  the 
world.  One  facilitates  interaction  within  a  simulated  physical  worid,  the  other  assists  .social 
interaction  through  the  creation  of  signs  and  cards.  They  each  borrow  their  appeal  through  their 
different  connections  to  real  worid  interactions.  Further,  they  each  appear  to  be  the  most 
successful  wiiliia  their  genre  of  software  because  they  maintain  fidelity  to  the  previously 
experienced  goals  and  environment  of  their  interactive  premise. 


*This  was  revealed  through  ihc  group  discussions  discussed  in  the  next  section 
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What  might  an  interactive  genre  look  like? 

If  children  are  given  an  opportunity  to  create  a  program,  they  will  create  one  which  is 
consistent  with,  and  motivating  because  of,  the  interactive  genre  which  serves  as  its  premise.  To 
explore  whether  students  would  emphasize  a  coherent  set  of  interactive  elements  and  what  those 
might  l)e,  the  two  autliors  conducted  group  interviews  with  two  different  sets  of  five  children.  The 
question  which  anchored  the  discussion  was,  "If  you  were  to  make  a  computer  program,  what 
would  it  be  like  and  what  sorts  of  things  would  you  include?" 

The  responses  of  both  groups  were  strikingly  similar  in  their  preference  for  adventure 
games.  Each  group  distinguished  these  from  mysteries  and  puzzles.  These  results  are  in 
accordance  with  the  distinctions  uncovered  by  the  clustering  task  and  the  motivational  rankings. 
By  looking  at  the  different  suggestions  the  students  made,  we  get  a  better  sense  of  the  coherence 
of  the  interactive  model  and  collect  evidence  for  an  extorsive  definition  of  the  adventure  genre  of 
interactivity. 

The  children  slated  that  they  wanted  an  impaclable  environment  We  might  call  this  a 
causally  coherent  model  of  interactivity  in  which  consequences  must  reveal  themselves  through 
changes  in  the  environment.  All  actions,  student  and  computer,  must  be  similarly  constrained  by 
the  environment.  Decisions  that  are  made  within  the  program  should  change  the  envuonment  in  a 
causal  fashion.  The  legacy  of  these  changes  remain  in  play  throughout  the  course  of  the 
interaction.  For  example,  forgetting  to  plant  the  com  will  become  a  damning  mistake  only  when 
the  food  stores  run  out.  Students  pointed  out  several  times  that  they  did  not  want  the  program  to 
give  disengaged,  omniscient,  or  external  feedback;  all  consequences  and  computer  reactions 
should  be  a  natural  result  of  the  causal  environment  the  program  creates. 

A  second  interactive  model  prefered  for  the  adventure  game  is  a  spatio-temporal  model. 
For  example,  they  liked  an  ability  to  move  h;  any  direction  spatially.  This  can  be  contrasted  with 
programs  in  which  the  child  always  moves  forward.  However,  complete  freedom  of  movemCiU 
and  choice  is  not  a  superordinate  principle  of  interactive  design.  The  children  explicitly  slated 
that  ihey  did  not  like  the  capability  of  going  back  in  time  to  correct  a  mistake.  Ideally,  an 
adventure  environment  will  replicate  the  advantages  and  constraints  of  a  spatio-temporal  world. 

Students  repeatedly  staled  that  they  liked  to  explore  and  learn  from  their  options  and 
mistakes.  They  liked  having  to  discover  the  sub-goais  and  emergent  goals  that  must  be  satisfied  to 
achieve  the  larger  goal  of  the  adventure.  They  wanted  the  program  to  be  somewhat  unpredictable 
but  in  a  fashion  consistent  with  the  laws  which  govern  the  program's  environment  They  wanted 
optional  levels  of  difficulty  and  novelty.  In  oiher  words,  they  wanted  to  learn  through  interactions 
within  a  rich  causal  environment 

The  adventure  game  relies  on  three  basic  interactive  models  which  are  derived  from  the 
child's  interactions  with  the  physical  worid:  causality,  spatio-tcmporality,  and  exploration.*  The 
discovery  or  immersion  learning  slyle  they  have  chosen  finds  a  natural  model  within  theii  own 
lives  (Papcrt,  1980).  However,  it  is  important  to  note  that  this  is  only  one  genre  of  interactivity 
and  a  style  of  learning  which  grows  from  it  It  is  important  to  imagine  what  sorts  of  social 
interactivity  and  learning  styles  will  become  supportable  as  computer  power  becomes  more 
available  in  the  classroom. 

Direct'ons  and  concKusion 

The  methods  of  enhancing  motivation  within  different  interactive  settings  will  differ 
depending  on  the  goals  the  user  generally  attributes  to  certain  types  of  interaction.  Although  the 
goal  is  ultimately  to  leam  something,  the  use  of  interactivity  can  bring  a  much  larger  set  of  goals 
and  expectations  to  bear.  It  is  central  to  any  interactive  design  that  these  expectations  are  played 

*l}icrc  were  other  elements  of  advcntuirc  games  which  the  children  mentioned  such  as  codes,  fantasy,  and  being  able  to 
chooye  the  characteristics  of  their  surrogates.  There  were  also  things  which  the  children  Claimed  were  ummpo  taot. 
graphics,  higl)  acuon,  points,  winnmg,  and,  sound.  It  is  wonh  noung  that  some  of  the  thmgs  excluded  arc  precisely  those 
things  which  are  mandatory  for  the  success  of  other  genres  of  interactivity.  It  would  have  been  iniercsting  to  have  asked 
the  students  to  design  the  ideal  dnU  and  practice  program  as  a  souice  of  contrast. 
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upon.  As  a  first  step  in  delineating  the  nature  of  these  expectations,  it  is  worthwhile  to  think  of  the 
different  types  of  interactions  a  user  might  expect  within  a  given  genre  of  interactivity.  A 
practical  way  for  a  designer  to  follow  this  prescription  is  to  ask  herself  what  sort  of  interaction 
does  this  computer  instruction  seem  most  like  and  what  sort  of  role  does  the  computer  play  in  the 
interaction  (e.g.,  partner,  adversary,  game  board,  world,  referee,  reference  tool,  museum,  etc.). 
Subsequently,  the  designer  can  consider  what  things  make  this  sort  of  interaction  most  satisfying 
and  wodc  onward. 

The  unique  features  of  the  new  and  highly  plastic  interactive  technologies  have  not  been 
fully  explored,  either  theoretically  or  empirically.  Designers  currently  rely  on  models  of 
development  which  are  often  out-dated  or  prematurely  reductionistic.  Much  of  the  literature  on 
motivation  in  instructional  technology  attempts  to  include  all  designs  within  large  matricies  and 
continuums  of  factors.  They  do  not  lake  into  account  the  qualitative  differences  of  children's 
expectations  in  their  encounters  with  instructional  technologies.  Much  like  film,  theie  are  genres 
of  software  (in  use  and  developing)  which  c^ilalize  on  the  child's  motivations  in  fundamentally 
different  ways.  Our  future  research  will  attempt  to  prove  the  interrelation  of  interactive  format 
and  motivation  through  an  experimental  design.  However,  the  next  step  is  to  continue  the 
discourse  with  children  about  the  different  interactive  elements  and  their  groupings  into  genres.  It 
is  here  that  we  will  find  the  best  ingredients  for  coherent  and  motivating  design  recipes. 
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InstractloBal  Devtlopwest  and  Clagsroon  Technology: 
Prototypt  Classrooms  at  thf  Navy*s  Surface  Warfare  Offfcers  School 


In  1988.  lows  Stale  Univwiity's  College  of  Ediication  and  Media  Resour;es  Center  (Amci,  Iowa),  contracted  with  Otk  Ridge 
Associated  Universilies,  Inc.  (Oak  Ridge,  Tennessee)  to  develop  a  plan  U>  prmiote  the  increased  use  of  educational 
tcdmology  by  the  instructors  and  students  of  the  United  Stales  Navy's  Surface  Warfare  Officen  School  (Newport,  Rhode 
Island).  This  pqw  is  a  description  of  the  process  used  to  develop,  implement,  and  evaluate  Ais  plan  which  cloaely 
followed  traditioBal  instnictional  design  procedures.  Rrst,  a  needs  assewment  was  conducted.  Second,  a  design  plan  was 
developed  and  critiqued,  and  third,  prototype  systems  were  installed  and  evaluated. 

L    Needs  Assessnent 

The  needs  assessment  phase  of  this  project  had  three  componcnte.  Fint,  it  wu  necessary  for  program  planners  to  gain  an 
understamiing  of  die  orgamzatkm  of  the  Surface  Warfare  Officers  School  (SWOS).  Second,  die  SWOS  curncuhmi  was 
evaluated  to  determine  the  applicaWlity  of  different  instructional  technologia.  Also,  SWOS  instructors*  teaching  styles, 
competaicies,  and  preferences  were  identified.  Third,  SWOS  facilities  were  analyzed  so  that  recommendations  could  be 
made  that  were  relevant  to  the  classrooms  and  laboratories  used  by  the  SWOS  instmctors. 

The  Mission  and  Orgtinlsatloii  of  the  Surface  Warfare  Omcem  School 

The  Surface  Warfare  Officers  School  Command  (SWOS)  is  located  in  Newport,  RI.  Its  mission  is  to: 

Provide  the  Naval  Surface  Warfare  Force,  through  a  system  of  functional  training,  witii  officers 
professionally  qualified  to  serve  as  effective  naval  leaders  on  surface  warfare  ships  with  the  ultimate  goal  of 
Command-at-Sea. 

Serve  as  the  focal  point  for  development  and  integration  of  qualificadon  standards  and  functional  training  in 
support  of  the  csteblishcd  continuum  of  Surface  Warfare  Officer  professional  and  billet  specialty  training. 

The  SWOS  has  iSve  permanent  buildings  and  five  temporary  buildings.  These  buildings  serve  as  classrocmu,  as  offices,  and 
as  laboratories.  There  are  five  schools  at  SWOS  (figure  1): 

1 .  The  Division  Officer  Tr^niny  School  -  This  school  prepares  prospective  Surface  Warfare  Officers  for  their  fint 
assignmenU  by  providing  fundamental  instruction  in  basic  fleet  training.  This  training  cnphasizes  die 
knowledge  and  skills  required  to  assume  duties  as  a  Dividon  Officer,  Officer  of  die  Deck  (in  port).  Combat 
Information  Center  Watch  Officer,  and  Junior  Officer  of  the  Deck.  Additionally,  the  course  provides  a  foundation 
of  knowledge  in  combat  systems,  propulsion  engineering,  and  damage  control. 

2.  T>>^  Dq)artmcnt  Head  Traii][ing  School  -  This  school  prepares  mid-grade  Surface  Warfare  Officers  to  confidently 
execute  Jq>artmcnl  head  level  dudes  aboard  surface  ships.  Training  addresses  shipboard  eq^iipment  in  a  sys^ 
fashion,  streuing  equipment  interaction  an .  interdepciMJence.  It  also  provides  advuiced  training  in  managerial, 
operational,  and  technical  areas  needed  to  su^rt  an  officer's  auignments  to  t  "  -wnfic  dcfMrtment  m  a  given 
type  of  ship. 

3.  The  Prospective  rr>mm»ndmg  QfTtfCT      Pm«p<^tive  E«ect^zive  Officer  (KOmO)  Training  Sdwol  -  The  PCO 
courw 's  primary  mission  is  to  prqMue  line  officers,  eligible  for  Command-at-Sea,  lo  propvly  execute  dwi 
authority  in  surface  ships.  The  training  addresses  the  professional  aspects  of  taking  ships  to  sea  nd  cxecutmg 
dieir  battle  functions  as  uniis  in  the  tactical  forces  of  die  U.S.  Navy.  The  school  also  addresses  command 
responsibilides,  Mp  tactics,  techniques  for  evaluation  and  control  of  ships  systems  and  equipment,  current  fleet 
policies  and  practices,  and  specialized  information  ^)propriate  to  the  ship  type  to  which  sttidentt  are  ordered.  The 
PXO  curriculum  is  des  :gned  to  provide  the  Executive  Officer  with  an  improved  concept  of  supporting  the 
Commandmg  Officer  in  controlling  and  evaluating  the  performance  of  the  ship. 

4.  T>c  Dsmagt^  <^ff^tm1  Training  School «  Courses  in  this  school  are  to  provide  training  in  shipboard  damage 
control  procedures.  In-deodi  study  is  provided  in  damage  control  admimstration  and  training,  divisional 
administration,  damage  control  equipment  systems,  and  odier  damage  control  conccpte. 
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5.  ^rhe  Engineenng  SnecUltv  Training  ScIym\  -  This  Jchoor«  cour»e«  of  ftudy  prepare  jumor,  division  officer 
!cvd,  Surface  Wwfixe  OfRcm  to  execute  tfieir  prospective  Enfifleeiing  D^Nfftment  Divi»ion  Officer  duties  aboard 
surface  ships.  The  training  emphasizes  tecimicai  and  cquipmcat  d^i&lls  relwd  to  ths  rsspcctivc  prcj^iIsiaT;  plsnt 
lysiems  and  iheir  operation.  Instruction  in  Ae  buic  imncipies  of  el<«tricity  is  cmphasizej?.  The  training  is 
developed  towaid  an  understanding  of  prq)er  and  safe  propulsion  plant  operation. 

The  Curriculum  nn6  Instructor  Analysis 

In  order  to  understand  the  media  needs  of  SWOS  instructors  and  curriculum,  a  process  was  followed  Uiat  analyzed  these  needs. 
The  god  of  this  process  was  to  obtain  as  much  information  as  pcssiUe  about: 

1.  The  current  status  of  the  instructional  systems  used  at  SV/OS. 

2.  instructors*  expectations  for  media  systems. 

3.  Curriculum  activities  that  required  media  support. 

4.  Teaching  procedures  and  styles  requiring  the  use  of  media. 

Information  was  obtained  during  two  visiu  to  SWOS.  First,  an  initial  visit  was  made  to  Newport  and  the  Surface  Warfare 
Officers  School.  The  purpose  of  this  visit  was  lo  familiarize  the  project  director  with  SWOS,  and  to  es^plain  to  SWOS 
officers  the  pcoccAires  to  be  foUowed  by  the  Iowa  Stale  University  (LSU)  team.  A  short  visit  was  made  lo  eadi  of  Ae  five 
schools  of  SWOS,  and  brief  discussions  concerning  media  systems  were  held,  Mmy  classrooms,  offices,  and  laboratories 
were  also  visited. 

Next,  two  members  of  the  !SU  team  visited  SWOS  to  c<OT^)lcte  an  in-depth  evaluation  designed  to  determine  what  media 
equipment  items  were  needed  to  support  the  curriculum  and  instructors  of  SWOS.  This  visit  was  OTganized  to  inchide  five 
activities:  in^lqjth  discussions  with  SWOS  officers  and  staff,  a  written  SWC»  instructor  survey,  an  cvahiolion  of  the  SWaS 
curricuhmi,  class  observations,  and  a  solicitation  of  media  equipment  requesu  from  the  offiwn  and  staff  of  SWOS. 

Addtionally,  a  comprehensive  facilities  review  was  completed.  This  review  included  an  examinati<m  of  SWOS  classrooms, 
offices,  and  laboratories.  It  also  included  an  inspection  of  fioor  plans  and  room  schematics. 

DISCUSSIONS  were  held  between  the  needs  assessment  team  and  selected  SWOS  officers.  Ttesc  discussions  were  of  two 
types  -  general  ones  with  caitral  SWOS  %appvn  staff,  and  more  specific  discussions  with  officers  from  eadt  of  th^  five 
SWOS  schools.  These  discussions  were  organized  to  obtsm  as  much  infwmation  as  possible  abut  the  media  needs  of 
SWOS.  The  following  conchisions  were  derived  from  thftse  discussions: 

1.  Most  teaching  conducted  at  SWOS  was  teacher-centered.  In  othsr  words,  most  instruction  was 'lecture  with 
demonstrations**.  The  teaching  varied  somewhat  between  th-^  SWOS  schools,  but  most  classroom  instruction  was 
primarily  lecture  with  teacher-led  discussions. 

2.  Insbuctor  background  and  ex^serience  with  media  equipment  was  varied.  Sonie  instructors  were  competent, 
some  were  inexperienced,  and  mest  were  self-taught 

3.  Instnicton  prepared  their  own  te^^hing  materials.  Many  stated  that  they  would  have  liked  to  have  been  able 
to  prq>are  more  so^^ticated  teaching  matcriMs,  assuming  production  techniques  were  easy  to  lc«n,  and 
I^oductiofl  equipment  and  facilities  were  readily  available. 

4.  Instructors  stated  that  they  wouW  like  to  have  all  cl  ssrooms  equipped  with  a  similar  configuration  of  support 
equipment  so  teaching  would  not  ha^  e  to  be  modified  depending  on  which  classroom  an  instructor  was  assigned 
to. 

5.  Instructors  felt  the  following  four  kinds  of  audio-visual  equipment  were  necessary:  overhead  projectors,  slide 
projectoii,  16mm  motion-picture  projectors,  and  video  di^lays.  They  also  felt  that  computer  dismay  or 
projection  was  important. 

6.  Training  in  teaching  was  adfqiiatr,  but  training  in  the  use  of  media  was  minimal.  Most  instructors  had  learned 
on  their  own,  «nd  expressed  concern  about  the  instaUatbn  of  overly  complex  classroom  audio-visual  equipment 
They  felt  equiiMnent  should  be  durable,  reliable,  accessible,  and  easy  to  use. 

7.  Instructors  requested  equipment  for  video  recording  of  class  sessions  for  student  review  and  for  instructor  self- 
evaluation. 
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The  next  component  of  !he  rosewdi  visit  was  the  INSTRUCTOR  QUESTIONNAIRE.  This  scvcn-question  survey  was 
distributed  to  SWOS  officers.  Eighty-five  qucstimrniiires  were  returned  (fifuiet  2  A  3). 

As  pfft  of  this  questimmaire,  instructon  were  uked  "open-ended"  questions  about  the  need  for  dassroom  equ^ent  and 
media  production  facilities.  A  variety  of  re^woscs  woe  made.  For  exampk,  several  SWOS  instructors  Mentioned  that  Acy 
would  like  to  have  the  schools  chalkboards  retraced  with  dry-markcr  metal  boards.  Also  mentioned  was  the  need  for 
"remote  contror  of  projection  equipment  Several  instructon  felt  thai  conqniters  should  be  placed  in  classrooms,  snd 
computer  projection  equipment  should  be  available. 

When  asked  about  media  production,  94%  of  the  SWOS  instructon  ^  responded  to  the  questionnaire  feh  ^  additkmal 
media  production  support  was  needed.  Traditional  grsphics  production  wu  mentioned  oftcru  as  wu  a  need  for  video 
production  equipment  The  VHS  format  was  considerably  more  popular  than  any  of  the  oflicr  video  formats.  Transparency 
production  was  mentioned  most  often  as  a  need  that,  if  met,  would  have  the  most  immediate  impact  on  teaching. 

The  questionnaire  results  made  it  fairly  obvious  that  SWOS  instructors  wanted  improved  classroom  audio-visual  systems, 
ar  \  that  they  felt  additional  media  produciiwi  was  critical  to  their  success  as  teachers.  No  negative  comments  about  the 
need  for  in^)roved  atidio-vistial  systems  were  given. 

A  CURRICULUM  EVALUATION  was  also  conducted  during  the  research  visit  A  number  of  curriculum  plans  from  each  of  the 
SWOS  schools  were  reviewed  The  rcsulu  of  this  review  supported  mmy  of  &e  generalizations  obtained  during  other 
{^ases  of  the  visit  Specifically,  the  review  of  the  curriculum  generated  d»ese  observations: 

1.  Instruction  was  based  on  goals  and  objectives,  with  student  learning  outcomes  clcariy  stated 

2.  Student  guides  were  used  throughout  the  SWOS  curriculum.  Often,  students  followed  along  in  their  guides 
during  teaching. 

3.  Instruction  was  tcachcr-centcrcd.  Teachers  iecoircd  to  present  information,  and  discussed  with  students  to 
clarify  key  points. 

4.  During  classroom  instruction,  instructors  often  used  materials  produced  directly  from  student  guides. 
Specifically,  charu,  drawings,  and  grq)hs  were  copied  from  student  guides  and  converted  into  trwisparcncies  or 
2«  ^  2**  slides. 

CLASS  OBSERVATIONS  were  used  on  several  occasions  to  verify  the  conchisions  derived  from  other  portions  of  the 
research  visit  Several  generalizations  weie  worthy  of  note: 

1.  All  instructon  observed  were  knowledgeable  in  the  topic  they  were  presenting  and  were  skillful  kcturen. 

2.  M  instructon  who  were  observed  used  imtructional  nuueriils  during  ttieir  lecn       Transparencies  were  used 
most  often,  followed  in  frequency  by  real  objects  and  slides. 

3.  StudcnU  seemed  interested  during  the  lectures  and  inquisitive  during  discussions.  In  almost  all  classrooms 
obsav©d»tudcnU  were  using  a  number  of  handoutt  (charts,  mipa,  drawings,  study  gui^  Desks 
and  t^e  tops  were  used  by  sttidenu  to  spread  these  materials  in  front  of  them  so  Aey  could  be  easily  referred  to 
during  lectures  and  discussions. 

4.  In  a  number  of  instances  instructon  were  forced  to  use  audio-visual  equyment  inapprqpriately.  This  was 
because  screens  were  mounted  incorrccdy,  pieces  of  projection  equipment  were  place  in^jpropriately,  or  lenses  on 
projecton  were  not  of  the  pn^  focal  length.  In  most  cases  this  was  not  the  fault  of  the  initructor,  but  was  due  to 
the  configuration  of  the  classroom  or  the  limitations  of  the  audio-visual  equipment  available  to  dicm. 

5.  Audio-visual  equipment  pieces  were  a  critical  component  of  instruction.  Instructors  used  e<jpiipmcnt  and  seemed 
desirous  of  making  greater  use  of  equipment. 

REQUESTS  for  audio-visual  equipment  and  support  materials  were  supplied  to  the  ISU  team  by  SWOS  officers  and  staff. 
These  requests  were  m  the  form  of  lisu  of  equ^mieal  and  copies  of  brochures.  These  specific  requesu  were  rAcrgod  witfi  the 
mote  general  information  obtained  during  the  odier  phases  of  the  needs  assessment  visit  Most  notable  were  die  foUowmg 
requests: 

1.  Requesu  were  made  (or  media  production  equipment  so  instructors  could  easily  and  efficiaitly  prepare 
instn^tional  materials. 

2.  Requests  were  made  for  more  video  display^ojcclion  equipment  in  the  VHS  IfT*  cassette  format 


ERLC 


568 

564 


3.  RequefU  were  made  for  computer  dupUy^jection  eqmpmcnt  to  computer  output  could  be  used  during 
classroom  prescnutions. 


4.  Requesu  were  made  for  newer,  easier  to  use,  16mm  xojectow. 

5.  Requests  were  made  for  compter  software  so  instructors  could  prepare  grq>hlcs  materials  for 
classroom  use.  Macintosh  and  "Macintosh  look-alike"  systems  were  requested  most  often. 


The   Facilities  An«lyslf 


A  review  of  all  current  SWOS  classrooms.  suRwrt  spaces,  and  simulation  facilities  was  conducted  using  on-site  visits  and 
an  examination  of  floor  plans.  Most  classrooms  and  support  spaces  were  observed  during  instructional  use,  as  well  u 
during  non-usage  time.  The  intent  of  tfiis  faciUtics  review  was  to  identify  and  categorize  the  buic  types  of  instructional 
environments  found  at  SWOS.  Also,  the  review  vUit  was  to  identify  tfie  physical  and  slnicniral  restrictions  that  influenced 
the  placement  of  equipment.  The  results  of  the  facilities  analysis  were  organized  into  nine  categories: 


1.  Physical  layout  (size,  sh^,  and  scaling  orientation) 

The  various  SWOS  classrooms  were  grouped  into  five  basic  physical  layout  patterns  (four  basic 
categories  and  a  fifth  category  to  cover  one-of-a-kind  facilities).  The  four  basic  patterns  and  the  fifth 
miscellaneous  grouping  identified  were  (see  figure  4): 

CATEGORY  1.  A  rectangular  space  with  the  front  of  the  room  oriented  on  Ae  kmgcst  dimension. 
Generally,  there  were  two  entrances  witfi  windows  on  the  opposite  wall.  The  rooms  seated 
approximately  45  at  fixed  desksAablcs. 

CATEGORY  2.  A  nearly  square  fomiat  space  with  two  fixed  seating  areas  on  either  side  of  a  structural 
cohmm(s)  located  near  the  front  of  the  room.  One  entrance  was  located  at  the  rear  of  the  ^pace  and 
windows  were  on  one  or  both  sides  of  the  room.  The  rooms  seated  i^roximately  30-35  in  small  desks. 

CATEGORY  3.  A  square  format  with  small  dimensions  (approximately  16'  x  18*)  with  <Hie  seating  area. 
Onecntrance  was  located  at  the  rear  or  front  of  the  ipacc  with  windows  on  cwwalL  The  rooms  seated 
approximately  10-15  at  movable  tables  or  desks. 

CATEGORY  4.  A  rectangular  format  space  with  standard  classroom  dimensions  (qiproximalely  30'  x 
40')  with  two  seating  areas  and  with  tf»e  front  of  the  room  oriented  on  the  narrow  dimension.  The  rooms 
had  one  or  two  entrances  with  windows  on  one  wall.  The  rooms  seated  approxiniately  30-35  at  movable 
ubles. 

CATEGORY  5.  Miscellaneous  formats,  ranging  from  a  large,  three-bay  auditorium  to  small  simulation 
(6*  X  8*)  training  facilities.  Included  were  a  variety  of  sizes  and  shi^  which  varied  drama!icaDy  from 
categories  1,  2,  3,  or  4.  Orientation  and  seating  patterns  also  varied. 

2.  Lighting  and  lighting  control 

In  general  the  lighting  in  the  SWOS  instructional  spaces  was  recessed  fhiorescent  fixtures  that  were  ccmtroUod  in 
banks  (two  or  three)     on/off  svritches  near  entrance  doors.  A  few  rooms  had  some  fixtures  or  additional  Hghts 
on  dimmer  controls.  Most  ^>ace$  with  windows  had  hand  operated  darkening  drapes  for  control  of  na^ 

3.  Projection  screens  or  surfaces 

All  SWOS  classrooms  had  one  to  three  screens  located  on  keystone  brackcu.  Many  were  mounted  approximately 
12-15*'  from  the  ceiling  and  oriented  parallel  to  the  chalkboards.  The  screen  sizes  varied  from  70-  to  96-  and  were 
generally  of  the  matte  surface  type.  Mounting  location  and  front  <mentatk>n  varied  from  room  to  room. 

The  viewing  angle  and  screen  orientation  of  the  SWOS  classrooms  varied  greatly.  Generally,  the  screens  did  not 
aUow  for  the  proper,  optimum  viewing  of  screen  images  by  all  siudcnu. 

The  ceiling  height  of  SWOS  classrooms  varied  from  less  ihim  the  normal  8'  height  to  14*  high  in  specialized 
areas.  Generally,  the  ceiling  hei^t  was  detennined  by  ceiling  tile  of  the  T-bar  suspended  acoustic  type. 
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All  SWOS  clMsrooms  had  dulkboani  nirf aces  loctud  it  the  from  of  the  room.  Mr  y  tbo  had  fide-mounted 
boards.  The  type  of  board*  varied  from  bias*  alate  lo  ^cen  bwurd  wirfaccf,  all  of  which  utilized  chalk  materials. 
Tack  or  mounting  boards  were  located  in  some  of  me  classroom  spaces. 

4.  Electrical  power 

Most  SWOS  classrooms  had  electrical  outlets  distributed  on  all  wall  surfaces  at  the  normal  level.  Only  a  few 
spaces  had  additional  floor  outlets  near  the  room  fronu  for  overhead  projector  usage.  None  of  the  SWOS  spaces 
had  power  outlets  located  in  ceilings  or  at  work  counter  levek. 

5.  Equipment  controls  and  sound 

None  of  the  SWOS  classrooms  had  front-to-rear  wiring  or  conduits  for  projection  equipment  contiob.  A  few 
spaces  had  "temporary**  ceiling-hung  slide  projector  remote  control  cables. 

Sound  amplification  or  enhancement  existed  in  only  a  few  of  the  larger  SWOS  facilities. 

Many  Division  Officer  School  classrooms  had  been  wired  for  distribution  of  television  and  video.  These  spaces 
had  one  or  two  small  15"  monitor/receivers  mounted  in  the  front  of  the  room. 

6.  Structural 

Some  SWOS  classrooms  had  structural  columns  located  within  the  viewing  and  seating  areas.  These  structural 
members  limited  projection  and  instructional  activities. 

7.  Audio-visual  equipment 

The  audio-visual  equipment  inventory  of  SWOS  was  dated,  a  "mixed-bag-  of  makea  and  nwdels,  and  limited  to 
primarily  front  screen  projection  items.  Many  of  the  overhewl  projectors  had  "standard"  angle  lenses  and  did  not 
allow  for  the  projection  of  properly  sized  images  in  the  existing  classrooms  with  !heir  limited  front^of-the-roor 
operating  space.  Video  equipment  wtt  of  various  modeb  and  formatt,  also.  Only  one  "green  saecn"  video 
projector  in  a  specialized  computer  laboratory  was  in  use. 

8.  Local  production  suppoit 

Similar  to  the  audio-visual  equipment,  the  local  graphics  and  other  media  production  equq)ment  was  dated,  of 
i^ed  varieties,  and  unconnected  in  approach.  The  ciq;>abilities  and  availability  of  production  materials  ar^l 
equipment  varied  from  school  departnicnt  to  department  and  from  office  unit  to  office  unit 

9.  A  summary  of  facilities  limiutions  and  characteristics  observed  during  review  and  analysis  indicated: 

a.  Varieties  of  i^ysical  layouU 

Four  basic  classroom  layouts  existed  and  each  (along  with  th&  miscellaneous  layouts)  required  individual 
audio-visual  plans. 

b.  Ceiling  height 

The  limited  vertical  height  in  all  SWOS  classrooms  was  a  restricting  factor  in  designing  for  the  overtiead 
viewing  of  visual  instmctional  materials.  In  most  rooms  only  4«"  or  Icsa  was  available  for  unobstructed 
viewing  of  oveihead  images.  Two  options  existed  in  most  spaces:  (1)  structtrral  ceiling  height  changes 
andAJr  (2)  design  of  a  media  system  not  dependent  upon  large  oi-crtiead  vkual  images.  Some 
rooms/spaces  needed  larger,  electrically  operated,  and  ceiling  recessed  screens. 

c.  Structural 

Some  classrooms  and  specialized  spaces  had  structural  members  or  colunms  in  their  seating  and  viewing 
areas. 

d.  Orientation 

Several  of  the  SWOS  classrooms  had  their  front  orientation  on  the  longest  dimension  of  the  room. 

e.  Standardization 
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There  was  «  Uck  of  i und«diziiion  in  the  type  of  equqiment  md  processes  stvtilable  for  u»tn|  and 
producing  tudio-visusl  prescntjaions  tt  SWOS.  Much  of  the  existing  equipment  wu  out-daied  and  in 
need  of  replacement 

f.  Lighting  control 

The  anangcmcnt  of  the  lighting  fixtures,  controls,  and  window  dariccning  was  generally  found  to  be 
adequate. 

Some  areas  required  slight  modification  of  the  light  switching  of  the  room(s).  Axiditional  switches  and 
dimmer  controls  needed  to  be  added  for  instriictor  oonvemcnce. 

g.  Audio 

There  was  a  general  lack  of  equipment  and  systems  to  produce  or  piiyback  audio  in  d»  SWOS 
classrooms.  These  nr:>'3  included  audio  amplification  of  spoken  oc  recorded  sound,  enhancement  of  the 
sound  portion  of  videos  and  films,  and  the  ability  to  record  sound  in  the  rooms. 

Acoustically,  the  SWOS  instructional  spaces  were  adequately  treated  with  drapes  and  acoustic  ceiling  tile 
to  allow  '•normal"  classroom  sound  recording  and  playback. 


II.    The  Design  Plan 

A.    The  Design  Considerations 

Based  on  the  dau  collected  during  the  needs  assessment  phase  of  this  project,  the  following  concepts  were  identified  and 
used  to  develop  the  design  proposal: 

The  Prototype  Concept  -  Hojotype  is  defined  w  the  first  or  primary  type  of  anything.  Prototypes  are  models  or  originah. 
A  prototype  classroom  would  be  one  equipped  with  audio-visual  systems  before  other  dassrooms  were  similarly  equipped. 
At  the  Surface  Warfare  Officas  School  there  arc  dozens  of  classrooms.  Befofe  any  single  plan  for  ©quipping  these 
classrooms  with  audio-visual  equipment  is  implemented,  it  should  be  first  tested  as  a  prototype. 

The  design  plan  had  two  major  purposes.  First,  it  contained  recommendations  for  how  SWOS  dasnooms  and  production 
areas  should  be  equipped.  These  recommendations  were  both  general  and  specific,  and  were  uied  u  Oie  basu 
insullation  of  systems  in  all  SWOS  classrooms.  An  in^rtant  second  purpose  of  this  plan  was  iu  recommendation  for  the 
installation  of  prototypes  which  were  to  be  installed,  demonstrated,  and  tested  before  ottier  cUssrooms  wt^c  similarly 
equipped.  The  resulu  obtained  from  this  prototype  process  were  used  to  validate  the  general  Tooommendations  of  die  plan. 

The  80%  "  20%  Concept  -  Very  e«ly  in  the  planning  process,  it  became  ^)pa^cnt  to  all  concerned  that  recommendations 
needed  to  satisfy  die  audio-visual  needs  for  not  only  the  present  situation  A  SWOS,  but  also  for  die  predictable  future  needs 
ofSWOS.  As  a  result  of  this  realization,  the  "W*.  20%  Conc<^"w«fonnulated.  Stated  generally,  thb  concept  impUtd 
that  80%  o  f  the  recomsicndaticM  contained  in  the  plan  wouW  be  to  meet  the  current  requirementt  of  tl^  SWOS  curricutom 
and  faculty.  Specifically,  dus  meant  that  more  efficient  mednxls  of,  and  equipment  for,  slide,  film,  video,  and  transparency 
projection  would  be  recommended  foi  installation. 

It  was  also  apparent  Uut  new  and  different  kinds  cf  instructional  technology  siqjport  for  SWOS  were  needed  so  diat  the 
curriculum  and  die  instructional  strategies  were  not  constrained  by  dui  limiutions  of  die  availaUe  media.  Twenty  percent 
of  die  support  proposed  by  Uus  plan  wu  to  meet  predicuble  future  needs.  Examples  of  this  ktad  of  sqjport  were  the 
availability  of  easy-to-use  computer  projection/display  hardware,  and  acceu  to  video  production  oquipmeiu. 

Audio-visual  systems  should  not  dictate  teaching  style  or  curriculum  offerings.  Conversely,  a  lack  of  systems  •^'^^^ 
limitdietechnkpesusedby  a  teacher,  or  die  kmds  of  activities  off  Thc*^0%  - 

20%  Concept**  was  an  aoen^  to  strike  a  balance  between  cunent  and  future  needs, 

B.    The  Design  Recommi^adatioBS 

Based  upon  the  design  needs  identified  during  die  needs  assessment  phase  of  die  project,  seven  kinds  of  cspabilities  were 
identified  as  necessary.  These  needs  are  listed  bdow: 


"  Video  display 
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.  videocAisette  oUvbtck 

-  computer  projcctTon/diipUy  (compotiic  RGB,  COA,  and  EGA  c^>tWlity) 
Front  screen  projectk)n 

-  overhead  tnm«pw«ncy  projection 

-  slide  projection 

•  16mm  projection 

-  screen  i^aoement 

•  lighting  requirements  control 

-  Audio  reinforcement/supplemental 

-  reinforcement  of  die  audio  portion  of  video  and  films 

-  public  address  amplificatiofn/recording 

-  auxiliary  audio  playback 


Video  recording 

-  session  recording 

.  instructiHr  analysis 

-  "minor**  production 


-  Graphics  support  -  local  production 

•  transparency  production 

-  computer  grq>hics/simulation  development 
.  slide  production 

-  scanning  of  documents 

Student  support/suidy 

*  video  i^ayback  and  viewing 

-  printed  nuuerial  study 

-  micro-forms  use 


-  Instructor  stipport 


-  instnictor  oHice  use 

-  netwiMrking  with  other  areas 

.  classroom  presentation  development 


There  were  five  plans  wrinen,  one  for  e«A  of  the  five  categories  of  classrooms.  The  five  prcsmtalkm  systems  eadi  dtowed 
forAeditt)UyofcompoiiKEGA,ROB.andCGA  video  images  from  a  variety  of  somas,  the^j^  of  stendard  pro 
inuges,  audio  playback  «dA*TOplificatton,  and  the  potential  AO  ttic 

nesentation  systems  utilized  modular  components,  hard  (fixed)  wired  remote  controls,  mobile  projecuon  cart^ctftoeu,  a 
fixed  control  console,  and  pr<^  screen  placement  for  optimum  viewing  within  the  ro^    layout  bmitatioM.  Each 
category  of  rooms  was  designed  to  accept  video  recording  cameras  at  two  locations  (overhead  and  rear-of-the-room)  usisig  t 
portable  recording  control  cmisole. 

Media  Production  Centcw  -  Inslnictional  maieriaU  used  in  teaching  are  obtained  in  wje  of  two  way¥  -  OMmnerci^y  prepared 
materiaU  can  be  purchased,  and  maieriaU  can  be  prodiiced  by  person  CommeiciaUy  prepared  m^^ 

are  usuaUy  of  a  high  technical  ijuafiiy  and  aie  laefeired  by  many  educators 

the  cumcubm.  However,  commcfcially  prep«d  matmals  have  several  disadvantagea  associated  wiA  their  use.  Rwt,  tfiey 
arc  expensive  to  produce  and  to  purchase.  Second,  because  devdc^  of  commerdd  teachmg  malm^ 
seU  ihSr  mataials  to  the  largest  possible  number  of  users,  they  tend  to  produce  general  materials.  This  ^^J'^^^!^^  ^ 
the  items  may  not  be  exacUy  "on  target"  for  specific  curricuhim  objectives.  Last,  comnwraal  materials  are  often  difficult  to 


revise. 


Many  professirmal  educators  now  beUeve  thai  developing  and  using  locally  produced  materials  »  the  mo^  effective  and 
efficient  way  to  mediate  instruction.  Optimally,  mstnictors  should  have  support  '"iffT^fi^l^J^  Sjf 
persona  produce  griphks  materials,  develop  videos,  and  carry  out  all  otfier  clencal  fusi^  needed  by  ^J^^f^^' 
l^fortuwuely.  mos7institutions  do  not  have  the  financial  resources  to  hire  support  staff.  Instead,  professional  educators 
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•dvocitc  the  installttion  of  a  mcdit  center  in  all  schools.  This  media  center  would  be  equij^  with  easy-to-use  production 
cquiptnem,  but  witfi  fs^'  people  The  regulsr  teaching  ttsff  would  use  the  niedis  csiter  to  produ^  tfieir  own  cjaairoom 
materials,  rathsr  than  to  have  others  produce  media  for  them. 

A  local  media  production  center  is  a  facility  where  instructors  can  easily  design,  produce,  and  revise  a  variety  of 
instructional  materials.  A  media  production  center  should  be  equq)ped  with  easy-to-use  equipment  and  materials  that  permit 
the  instructor  to  qiuckly  and  professionally  develop  high  quality  teaching  media.  Production  centers  should  be  located  as 
close  to  instructors  as  possible,  and  the  more  often  a  technique  is  used,  the  more  convenient  it  should  be. 

Two  kinds  of  media  production  centers  were  needed  at  SWOS.  One  large,  centrally  located  center  was  needed  to  provide  the 
majority  of  support  to  most  Instructors.  A  second,  satellite  media  center  was  needed  at  the  Division  Officers  School 

1.  Tne  Satellite  Media  Production  Center  (SMPC) 

The  first  media  production  center  at  SWOS  was  a  prototype  satellite  center  located  in  the  Division 
Officers  School.  This  center  had  the  following  types  of  production  capabilities: 

a.  Manual  uid  mechanical  production  of  graphic  materials 

b.  Computer  production  of  graphic  materials 

c.  Simple  photographic  production 

d.  One-camera  video  production  and  editing 

Additionally,  the  SMPC  needed  equipment  so  thai  insinictors  could  preview  instructional  materials  that  had  been 
produced  or  for  reviewing  commercial  materials. 

2.  Central  Media  Production  Center  (CMPC) 

The  primary  media  production  ccntei-,  located  centrally,  had  the  same  capabilities  as  the  SMPC.  Additionally,  it 
had  the  following  capabilities. 

a.  Film  to  video  transfer 

b.  Copy  machine  with  enlarging  and  reducing  capabilities 

c.  Computer-based  instructional  material  production  using  various  computer  operating  systems 

d.  Video  format  dubbing 

C.    The  Design  Plan  Evatuatioa 

The  design  plan  evaluation  was  submitted  to  SWOS  instructo-s  for  their  review.  A  formal  presentation  was  given,  and  the 
specific  recommendations  for  prototype  instaUations  were  mkde.  After  the  presentation,  questionrodres  were  distributed  to 
obtain  opinions  about  the  proposal  Part  1  of  the  questionnaire  asked  questions  about  the  proposed  classroom 
installations.  Part  1  contained  questions  about  the  prototype  media  centers. 

Resulu  (see  tables  1,  2,  &  3)  indicated  that  the  evaluators  thought  the  plan  for  the  classrooms  was  compatible  with  SWOS 
needs,  had  qjpropriatc  ccnaalized  control  of  equipment,  and  met  expecutions.  The  level  of  sojAistication  of  the  system 
was  considered  "about  right".  The  evaluators  were  slighUy  less  positive  about  the  proposed  media  centers.  However, 
ratings  were  generally  very  favorable. 

Ill,     Prototype  Installations 

Because  evaluations  were  so  positive,  few  modifications  to  the  proposed  plan  were  made.  Only  minor  alterations  to  the 
recommendations,  such  as  the  placement  of  equipment  in  the  classroom  media  control  system,  were  made. 

Six  prototype  systems  were  completed.  One  of  each  of  the  first  four  categories  of  classrooms  had  equipment  installed  in 
them.  Also,  the  prototype  satellite  and  central  media  centers  were  equipped. 


A.  Category  1  Classroom  (figures  5  &  6) 
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The  prototype  clMsroom  tystcm  design  for  category  #1  cla»«rooms  incorportted  the  general  design  functions  as 
follows  (sec  figures  7  -  ii): 

L  Displaying  video  images  using  a  ceiling  mounted  multi-scan  projector,  supplemented  by  the  use  of 
two,  fhmt-comcr-of-room  niounted  video  monitors. 

2.  Displaying  film,  slide,  and  overhead  transparency  images  using  a  rear-of-roora  mobile 
projection  cart  and  front-of-room  inslnicior  workstation. 

3.  Providing  for  audk>  reinforcement  using  a  supplemental  public  address  amplificr/mixci',  sound 
input  panel  and  player,  auxiliary  wall  speakers,  and  the  video  monitor's  built-in  sound  systems. 

4.  Allowing  for  video  display  and  recording  using  a  two-camera  recording  system  (overhead  and  rcar-of- 
the-room  k>cations)  with  an  insiruc'xw  controlled  console.  The  system  (cameras  and  ooniole)  was 
designed  to  be  partially  mobile.  The  camera  mounu  and  wiring  were  fixed  in  each  classroom  space,  but 
the  csneras  and  control  console  were  a  pwtable  unit  that  could  be  moved  ftom  classroom  to  classroom. 

5.  Providing  for  instructor  control  and  operation  of  all  audio-visual  equipment  using  a  tinree  part 
(audio,  video  playback,  and  video  recording)  console  system.  All  power,  lighting,  and  equipment 
controls  were  built  into  the  console  and  wired  pfjrmancntly  to  the  A-V  items  or  patch  boxes  at  the 
equipment  sites. 

6.  Providing  three  screens  ,  properly  mounted  and  angled,  to  allow  viewing  by  as  many  snidents  as 
possible. 


B.  Category  2  Classroom  (see  figure  12) 

The  prescnution  sys»;em  plan  for  category  #2  classrooms  incorporated  the  general  design  functions  as  follows: 

1.  Displaying  video  images  using  three  video  moniton  (two  front-comer  mounted  and  one 
nvounted  on  Oie  center  structural  column). 

2.  Displaying  film,  slide,  and  overhead  transparency  images  using  equqment  mounted  in  a  cabinet 
on  the  center  structural  colunm,  axul  a  cart-mounted  overhead  projector. 

3.  Providing  for  audio  reinforcement  by  using  a  public  address  amplifierAnixer,  sound  iiqwt  panel,  audio 
player,  auxiliary  wall  speaken,  and  the  video  monitor's  sound  system. 

H  Allowing  video  display  and  recording  by  using  a  two-camera  recordist  system  (overhead  and  rear- 
of.  ihe-room  locations)  with  an  instructor  controUed  console.  The  system  .cameras  and  console)  was 
desi^Tied  to  be  mobile  witfi  the  camera  mounU  and  wiring  fixed  in  each  ciassroom  space  and  die  cameras 
and  c-introl  console  a  portable  unit  that  could  be  moved  from  classroom  to  classroom. 

5.  ?r  ^viding  for  instnictor  control  and  operation  of  all  audio-visual  equipment  by  the  use  of  a  three  part 
(audi  s  video  playback,  and  video  recording)  console.  All  power,  lighting,  and  individual  equipment 
conti  i\s  were  built  into  the  console  and  wired  permanently  to  die  individual  A-V  items  or  patch  boxes. 

6.  Providing  two  screen  surfaces,  properly  mounted  and  angle<^  to  allow  efficient  viewing  by  as  many 
students  as  possible. 

C.  Category  3  Classroom  (see  figures  13  &  14) 

The  prototype  presenution  system  plan  for  the  category  #3  classrooms  incorporated  the  general  design  functions 
as  follows: 

1.  Displaying  video  images  (composite  -  EGA)  using  a  front-comer  mounted  video  monitor. 

2.  Displaying  film,  slide,  and  overhead  transparency  images  using  a  rear-of-room  mobile  projection  cert 
and  front  instructor  worksution. 

3.  Providing  limited  audio  reinforcement  and  playback  using  a  sound  input  panel,  audio  player,  and  the 
video  monitor^s  sound  system. 
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4.  AUowmg  for  video  diiplty  «id  recording  by  using  a  iwo-ctmem  recording  system  (overhead  and  rw- 
of-the  room  locations)  with  m  instructor  controlled  console.  The  system  (cameras  and  console)  was 
designed  to  be  mobile  with  tfie  camera  mounts  «ul  wiring  fixed,  and  the  cameras  and  control  console  as  a 
portable  unit 

5.  Providing  for  instnictor  control  and  operation  of  all  audio-visual  equipment  by  using  a  three  part 
(audio,  video  playback,  and  video  recording)  console.  All  power,  lighting,  and  individual  equipment 
controls  were  built  into  the  console  ar.d  wired  permanently  to  the  individual  A-V  items  or  patch  boxes. 

6.  Providing  two  screen  surfaces,  properly  mounted  and  angled,  to  allow  viewing  by  as  many  students  as 
possible  within  the  limits  of  the  physical  environment 

D.  Category  4  Qassroom  (see  figure  15) 

1.  Displaying  video  images  using  two,  front-comer  moxmtcd,  video  monitors. 

2.  Displaying  film,  slide,  and  overhead  transparency  images  by  using  a  njar-of-room  mobile 
projection  cart  tnd  front-of-room  instructor  worksuiion. 

3.  Providing  for  audio  reinforcement  by  the  use  of  a  public  address  amplifier/mixer,  sound 
input  panel,  audio  player,  auxiliary  column  wall  speakers,  tnd  by  using  the  video  monitor's  sound 
system. 

4.  Allowing  video  display  and  recording  by  using  a  two  camera  recording  system  (overhead  and  rear- 
of-room  locations)  with  an  instructor  controlled  console  located  in  the  front  of  the  classroom.  The 
system  (cameras  and  console)  was  designed  to  be  moHlc  wifli  the  camera  mounts  and  wiring  fixed  and  the 
cameras  and  control  console  a  portable  unit  that  could  be  moved  from  dassnxMn  to  classroom. 

5.  Providing  for  instnictor  control  and  operation  of  all  audio-visual  equq)ment  by  usmg  a  diree  part 
(audio,  video  playback,  and  video  recording)  console.  All  powcr^  lighting,  and  individual  equipment 
controls  were  built  into  the  console  and  wired  permanently  to  Ac  individual  A-V  items  or  patch  boxes. 

6.  Providing  two  screen  surfaces,  properly  mounted  and  angled,  to  allow  efficient  viewing  by  as  many 
smdents  as  possible. 

E.  Satellite  Media  Production  Center  (SMPC;  see  figure  16) 

One  concern  expressed  numerous  times  by  the  officers  and  staff  of  SWOS  was  the  need  for  ccmvcnicnt,  easy-to-use, 
media  production  centers  where  they  could  prepare  professional  appearing  instructional  materials.  In  order  to 
meet  die  needs  expressed  by  SWOS  officers  for  media  support,  two  prototype  media  production  centers  were 
installed,  demonstrated,  and  evaluated. 

The  SMPC  had  materials  production  and  preview  cqwbilitiea.  Production  capabilities  mcluded  those  for 
graphic,  computer,  photographic  and  video  development  Equipment  items  for  previewing  media  were 
also  installed. 

1.  Py^yinction  r^abilities  -  Production  of  graphic  materials  required  *hc  installation  of  the  following 
types  of  equipment: 

a.  Mechanical  lettering  machine 

b.  Layout  table 

c.  Thermal  transparency  maker 

d.  Slide  previewing  and  tracing  light  Ubie 

e.  Laminator 

The  following  computer  equipment  was  included  in  the  SMPC: 

a.  A  computer  with  *\jser-firicndly- software 

b.  A  scanner  for  digitizing  printed  materials  such  as  charu  and  line  drawings 

c.  A  laser  printer 

The  following  jAotogn^hic  equipment  was  placed  iii  the  SMPC: 
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t.  Stngk  lens  reflex  camera 

b.  Copy  tund  (for  making  tlidea  firom  flat  pictures) 


The  following  video  equipment  was  placed  in  the  SMPC: 

a.  Portable  VHS  video  recorder  with  camera 

b.  VHS  1/2**  videocassett  editing  system 

0  Prwiew  Capabilitiei: 

a.  A  carousel  slide  projector 

b.  Overhead  projector 

c.  A  VHS  1/2"  videocasscttc  player  with  TV 

R  SWOS  Central  Media  Ptoduction  Center  (CMPC;  see  figure  17) 
This  center  was  die  primaiy  media  production  center  for  SWOS. 

Capabilities  -  The  CMPC  had  five  categories  of  cqwbilities.  First,  it  was  a  media  production  center  for  SWOS 
instructon.  It  had  the  same  capabilities  as  the  Satellite  Media  Production  Center.  Second,  it  contained  one-of-a- 
kind  media  production  items  that  were  needed  but  not  routinely  used  For  example,  video  equ^ment  in  the  non- 
domestic  PAL  and  SECAMfwmals  were  kq)t  in  the  CMPC.  Third,  "back-up"  equipment  wMkqH  here.  Fourth,  the 
CMPC  was  where  software  and  supplies  needed  by  SWOS  instructors  was  stored.  Last,  the  CMPC  contained  a  study 
area  for  SWOS  students. 

G.  Prototype  Evaluation 

Insudladon  of  the  six  prototypes  was  completed  by  Iowa  State  Univeni^  stafif  duting  three  visits  to  SWOS. 
When  all  systems  were  fully  operational  a  formal  presentation  of  die  systems*  ci^Mbilities  was  made  U>  SWOS 
commanding  officers.  At  d»c  conchukm  of  this  presentadon,  die  evaluation  of  die  newly  installed  prototypes 
began.  Officen  weie  given  a  questionnaire  to  conq>lete  after  they  had  a  chance  to  use  one  or  more  of  die 
prototypes.  These  questionnaires  were  collected  several  weeks  later.  Generally,  results  were  favorable  (see  table 
3).  Suggestions  for  improving  the  prototypes  inchided: 

1.  Relocate  equipment  controls  in  die  console  so  diey  were  easier  to  use. 

2.  Use  of  wireless  microjrfiones  was  not  a  good  idea;  use  a  microphone  widi  a  caWe  or  do  not  use  at  all. 

3.  Install  5  1/4"  disk  drives  in  die  prototype  system's  microcomputers.  3  1/2"  disks  were  noC  installed 
in  any  of  die  command's  odier  computers. 

4.  Put  a  dot  matrix  printer  in  the  media  centers. 

5.  Add  a  spotlight  in  clusrooms  to  illuminate  overhead  camera  work. 

Generally,  die  evaluation  indicated  dut  SWOS  officers  were  satisfied  widi  die  process  followed  and  die  equipment 
installed  during  this  project 


Summary 


Instructional  design  (ID)  procedures  were  followed  during  diis  project.  Traditionalists  normally  limit  die  application  of  the 
ID  process  to  die  development  of  instructional  materials,  or  instructional  sequences.  This  project  applied  ID  procedures  to 
die  problem  of  providing]  qjpiopriate  hardware  support  for  die  five  schools  of  die  Navy's  Surface  Warfare  OfRcfxs  School. 
It  was  fbund  dut  die  dcvign  presci^Hions  of  ID  dieoiy  were  just  as  i^ipropriate  for  a  hardwve-based  project  as  diey  were  for 
software  development. 
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Figure  2 
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Figare  3 
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Figure  4 
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Figure'  5 


Category  #1 :  Classroom 


-A 


□ 
9 


Front 


8 


□ 

9 


1 .  Video  Projector 

2.  Slide/film  Projection  Console 

3.  Overhead  Projection 

4.  Screens,  a  &  70" 

5.  Video  Receiver/monitor 

6.  Control  Console,  19": 

sound/PA 
slide/film  remote 
video  proj.  remote 
video  camera  controls 
VCR 
conputer 

7.  Video  Camera  Locations 

8.  Instructor  Work  Station 

9.  Audio  Speakers 
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Figure  6 


Figure  7 


2.1   INSTRUCTOR  CONSOLE  -  COHPONENTS 
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Ficure  t 


2.2   CLASSROOM  DISPUY/PROJECTION  •  COHPONEHTS 


AA  -  Projector(s)  remote  control/audio  inputs 

BB  -  AC  switched  power  -  To  switch  (B)  on  front  console 

CC  -  Slide  projectors 

DD  - 16  mm  projector 


ERIC 


5S1 

584 


Fifvre  9 


2.3  CUSSROOH  VIDEO  DISPLAY  AMD  RECOROINfi  -  COMPONENTS 


Figure  10 


COHMHEHIS  lOENTIFICATlM  -  SUOS  ClASSROOH  AUCIO-VISUAL  SYSTEM 
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Figure  11 
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Figure  12 
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1,  Video  Monitors 

2,  Slide/film  Projection  Systems 

3,  Overhead  Projection  & 
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Video  Camera  Location 
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Category  #3:  Seminar  Room 


Front 


1.  Video  Monitor 

2.  Slide/film  Projector  Console 

3.  Overhead  Projector 

4.  Screens,  70"  &  84" 

5.  instructor  Work  Station 
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7.  Video  Camera  Locations 

8.  Audio  Speaker 


583 


Figure  14 


Figure  15 

CATEGORY  4. 


1.  Video  Monitors 

2.  Slide/film  Projection  Systems 

3.  Overhead  Projection  & 

Instructor  Work  Station 

4.  Control  Console 

5.  Video  Camera  Location 

6.  Audio  Speakers 
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Figure  16 
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Figure  17 


Prototype  for  the 
Central  Media  Production  Center 
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5.  Digitizer 

6.  Extensior^  Drive 
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Table  1:    Design  Plan  Evaluation  •  Classroom  Systems 


Average  Rating 

A.*         1.  Proposal  systems  seem  compaiible  with  classrooms?  1.9 
1  The  centralized  control  console  for  the  system  is  appropriate?  2.23 
3.  The  proposed  design  plan  meets  my  expectations?  1  -54 

B  ♦*       The  level  of  sophistication  of  the  proposed  systems  seems:  6.3 1 

C.***      Overall,  I  feel  classroom  presentation  needs  are  being  met  2.09 


♦  14  Point  Likcrt  Scale  Used 

1  =  strongly  agree 

4  =  agree 

10  =  disagree 

14  =  strongly  disagree 


♦*  14  Point  Likert  Scale  Used 

1  =  too  high 

4  =  high  but  reachable 

7  =  about  right 

10  =  low 

14  =  too  low 


14  Point  Likert  Scale  Used 
1  =  very  adequately 
4  =  adequately 
10  =  inadequately 
14  =  very  inadequately 
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Table  2:    Design  Plan  Evaluation  •  Media  Centers 


Avcragg  Rating 

A.*         1.  The  proposal  appears  to  provide  adequate  support?  4.0 
2.  The  proposed  design  plar  meets  my  expectations?  2.75 
B  **       The  level  of  sophistication  of  the  proposed  systems  seems:  6.17 
C.***      Overall,  I  feel  media  production  needs  are  being  met  2.75 


*  14  Point  Likert  Scale  Used 

1  =  su-ongly  agree 

4  =  agree 

10  =  disagree 

14  =  su-ongly  disagree 


14  Point  Likert  Scale  Used 
1  =  too  high 
4  =  high  but  reachable 
7  =  about  right 
10  =  low 
14  =  too  low 


**♦  14  Point  Likert  Scale  Used 
1  =  very  adequately 
4  =  adequately 
10  =  iiiadequately 
14  =  very  inadequately 
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TabN**  3:    Evaluation  of  the  Four  Prototype  Classroom  Installations 


Question 

Average  Ratine 

1.  Layout  of  the  console  is  logical. 

4.2 

2.  Equipment  pieces  are  e^^sy  to  use  and  control. 

3.2 

3.  The  prototypes  perform  adequately. 

2.9 

4.  The  range  of  functions  meets  my  expectations. 

2.4 

5.  The  prototypes  arc  a  significant  improvement  over  previous  systems. 

2.0 

0.  1  nc  ciassrooni-uscrs  nianuai  is  ubciui. 

2.6 

How  often  do  you  expect  to  use: 

1.  Screen  projection  equipment 

1.4 

2.  Audio  playback  and  recording 

5.6 

3.  Video  playback 

2.8 

4.  Video  recording 

3.3 

5.  Computer  display 

5.1 

*  14  Point  Likert  Scale  Used 

I  =  strongly  agree 

4  =  agree 

10  =  disagree 

14  =  strongly  disagree 


**  14  Point  Likert  Scale  Used 

1  =  high  use 

4  =  occasional  use 

10=  seldom  use 

14  =  never  use 
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THE  CORPORATE  COLLEGE: 
Integrating  Infonuatlon  Technologies 


Summary 

During  the  year  since  the  founding  of  Squibb  College, 
the  Instructional  Technology  department  has  been 
charged  with  planning  and  implementing  information 
technology  systems  for  both  office  productivity  and 
training.     Decisions  made,  obstacles  encountered,  and 
progress  achieved  during  that  year  are  described. 

The  impact  of  the  first  year-s  experiences  on  future 
decisions  is  discussed.    Major  aspirations  and 
challenges  for  the  coming  years  are  also  outlined. 


Background 

Founded  in  September  1988,  Squibb  College  was  created 
by  E.R.  Squibb  &  Sons  to  meet  the  basic  training, 
education,  and  development  needs  of  its  more  than 
20,000  employees  worldwide.    The  initial  mission 
statement  of  the  College  stressed  the  corporation's 
commitment  to  state-of-the-art  technologies. 


The  Squibb  College  will  be  the  corporate 
umbrella  for  all  training,  education  and 
development  activities  worldwide. 

The  College  will  initiate  and  design 
management  and  career  development  programs  as 
well  as  support  special  focus  programs 
related  to  specific  business  units  and 
functional  responsibilities. 

Squibb  College  will  ...  integrate  a  worldwide 
state-of-the-art  training  technology  .... 

The  College  is  dedicated  to  supporting 
corporate  goals  by  meeting  the  learning  needs 
of  its  employees,  enabling  them  to  innovate 
and  excel  .... 


—  Squibb  College  Mission  Statement,  1988 


Instructional  Technology 

To  help  fulfill  that  mission,  the  College  established 
the  Instructional  Technology  department,  the  first  such 
function  in  the  corporation.    The  mission  of 
Instructional  Technology  is  to  serve  the  instructional 
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systems  development  (ISD)  needs  of  the  College's 
curricular  departments,  such  as  Career/  Management 
Development.    We  also  provide  ISD  services  to  various 
units  outside  the  College  organization,  such  as 
Pharmaceutical  Group  Sales  Training. 

Prior  to  the  advent  of  the  Instructional  Technology 
department,  training  program  directors  had  relied  on 
external  vendors  and  consultants  for  all  training 
product  development  and  for  most  design  and  delivery 
requirements.    There  had  been  no  internal  instructional 
developers  and,  thus,  were  no  ISD  tools.    The  new 
department  was  starting  "from  scratch." 


The  Technology  Challenge 

The  Instructional  Technology  department  did  not  really 
launch  its  activities  until  January  1989,  just  after 
Squibb  College  moved  into  a  renovated  training  facility 
north  of  Princeton,  New  Jersey.    At  that  time,  we  were 
assigned  the  task  of  integrating  information 
technologies  into  the  training  development  and  delivery 
processes  at  Squibb  College. 

The  creators  of  Squibb  College  agreed  with  Weiss  and 
Birnbaum  that  "technical  infrastructure  is  as  important 
as  communications  and  transportation  in  constraining  or 
facilitating  business  activity"   (1989,  p. 1015).  Thus, 
because  we  were  "the  technology  people"  for  the 
College,  coordinating  the  selection  and  implementation 
of  office  automation  systems  was  also  among  the 
responsibilities  given  to  Instructional  Technology. 


Integrating  Technologies  for  Productivity 

When  Squibb  College  first  moved  into  its  facility, 
fewer  than  half  of  the  staff  of  20  was  equipped  with  an 
office  microcomputer.     The  rest  of  the  staff  were 
recently  hired  to  fill  brana-new  positions  and,  thus, 
brought  no  computing  resources  with  them.  Therefore, 
the  first  major  challenge  for  the  Instructional 
Technology  group  was  to  select  and  provide  essential 
productivity  tools  for  all  College  personnel,  ensuring 
insofar  as  possible  that  the  tools  we  r^hose  would  serve 
the  College  well  for  up  to  three  years. 
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Decisions 


Long-range  planning  does  not  deal  with  future 
decisions,  but  with  the  future  of  present 
decisions. 


—  Peter  Drucker 

It  was  immediately  evident  that  much  time,  energy,  and 
paper  could  be  saved  by  linking  everyone  through  a 
local-area  network  (LAN).     For  example,  we  estimated  a 
time  saving  of  70%  in  scheduling  meetings  alone, 
through  the  use  of  the  "Scheduler"  feature  of 
WordFfa;rfect  Office,  the  corporate  standard  for  LAN- 
based  menu  interface. 

Hardware  decisions  were  fairly  easy,  because  few 
options  were  available  to  us  within  the  limits  of  the 
corporate  standards  laid  down  by  Squibb 's  Information 
Resources  group.     For  example,  the  LAN  specifications 
were  predetermined.    We  did  select  a  basic  desktop 
system  for  each  staff  member  and,  for  users  whose 
activities  demand  greater  computing  power  or  greater 
variety  of  capabilities,  a  range  of  options.    All  of 
these  hardware  selections  are  shown  in  Table  1. 

(Table  1  about  here) 

Based  on  a  careful  task  analysis  of  Squibb  College 
activities,  standard,  LAN-based,  productivity  software 
was  selected  from  those  packages  formally  approved  as 
corporate  standards.     This  was  a  more  difficult,  more 
time-consuming  analysis,  encompassing  as  it  did  several 
departments  varying  greatly  in  the  complexity  of  their 
needs.    The  range  of  basic  software  purposes,  our 
salected  tools,  and  typical  College  applications  are 
summarized  in  Table  2. 

(Table  2  about  here) 


Implementation 


The  art  of  progress  is  to  preserve  order  amid 
change  and  to  preserve  change  amid  order. 


—  Alfred  North  Whitehead 

Because  of  Squibb 's  extensive  request-approval- 
implementation  process,  we  implemented  our  productivity 
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technologies  in  two  manageable,  orderly  phases;  first, 
stand-alone  systems;  second,  the  LAN. 

Implementing  stand-alone  systems  first  allowed  College 
personnel  to  begin  learning  fundamental  computer 
operations  and  software  packages  almost  immediately,  so 
that  they  could  become  productive  quickly*    To  avoid 
unnecessary  duplication  of  expenditures  for  both 
single-user  and  network  versions  of  software,  at  this 
stage  we  provided  single-user  copies  of  only  the  most 
basic  software  packages  -  WordPerfect.  Lotus  1-2-3.  and 
a  few  special  packages  for  certain  users. 

As  it  happened,  our  implementation  plan  was  fortuitous. 
A  sequence  of  approval  delays  pushed  the  implementation 
of  the  LAN  back  to  December  1989/January  1990.     Had  we 
elected  to  implement  the  network  immediately,  we  might 
have  been  without  computational  power  for  a  year! 


Integrating  Technologies  for  Training 

The  second  major  challenge  for  Instructional  Technology 
was  planning  and  implementing  the  rational ^  integration 
of  technologies  into  training  curricula,  with 
particular  emphasis  on  Pharmaceutical  Group  Sales 
Training. 


Decisions 

In  the  beginning,  most  instruction  at  Squibb  College 
was  traditional  "stand-up"  training,  using  flipcharts, 
overhead  and  slide  projectors,  and  linear  videotapes. 
Trainers  had  occasionally  employed  a  variety  of  other 
technologies:     optical  scanners,  interactive  keypads, 
computer-based  testing.     Sales  trainers,  in  particular, 
were  pleased  with  the  versatility  provided  by  computer- 
controlled  media,  and  were  eager  to  integrate 
additional  innovative  technologies  into  their  training 
programs. 

Of  all  instructional  technologies  available  early  in 
1989,  the  greatest  learning  gains  and  instructional 
variety  were  achievable  through  interactive  videodisc. 
Information  Resources  had  a  corporate  standard  for 
that,  too:     IBM  InfoWindow  units  using  SONY  LDP-2000 
laser  videodisc  players.    Wc  therefore  designed  an 
Interactive  Learning  Center  at  Squibb  College  that 
would  consist  of  eight  to  twelve  networked  IBM  PS/2 
Model  50Zs,  with  InfoWindow  touch-screen  monitors  and 
SONY  videodisc  players.     Other  peripherals,  such  as 
videotape  players  and  interactive  keypads  would  be 
rtontrolled,  as  desired,  by  the  PS/2  computers. 
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Dissemination  of  independent-study  programs  to 
employees  at  other  Squibb  sites  was  also  a  concern  for 
Instructional  Technology.    We  recommended  that  each 
sales  distribution  center  and  each  major  employee 
center  in  the  USA  be  provided  with  two  InfoWindow  units 
that  would  run  the  programs  we  were  creating. 

Squibb 's  European  operations,  however,  could  not  use 
IBM  InfoWindow  systems,  because  the  NTSC  video 
convention  used  in  the  USA  is  incompatible  witn  the  PAL 
and  SECAM  video  conventions  used  in  Europe.     For  these 
operations,  we  were  initially  forced  to  consider 
transferring  programs  to  other  media,  such  as  simple 
computer-based  training  or  linear  videotape. 
Fortunately,  dual-standard  interactive  systems  are  now 
available,  so  the  sharing  of  training  programs  with  our 
colleagues  is  no  longer  an  insurmountable  problem. 

The  selection  of  ISD  software    as  been  more  difficult. 
Certainly,  our  choice  of  delivery  system  somewhat 
limited  our  software  options.     Early  in  1989,  only 
InfoWindow  Presentation  System  (IWPS)  and  Learning 
System  1  (LS/1)  authoring  systems  were  available  for 
IBM  InfoWindow  (of  these  two,  we  preferred  IWPS. 
although  neither  met  our  needs) . 

Also  limiting  were  the  software  choices  made  by  the 
vendors  who  created  programs  for  Squibb  College, 
because  we  would  have  to  be  able  to  revise  and  update 
such  programs.    For  example,  an  extensive  computer- 
based  testing  program  was  prepared  using  TenCcre;  a 
career  development  program  was  created  using  Summit;  an 
employee  information  program  was  designed  with  IWPS. 
We  realized  that  we  would  have  to  purchase  all  of  these 
authoring  systems  (for  maintenance),  as  indica-^.ed  in 
Table  3 

(Table  3  about  here) 

As  the  number  and  capabilities  of  authoring  systems  has 
increased.  Instructional  Technology's  task  has  become 
simultaneously  easier  and  more  difficult.    We  can  now 
find  authoring  systems  with  the  functions  we  desire; 
however,  these  systems  may  not  run  on  our  corporate 
standard  hardware.    We  are  still  evaluating  authoring 
software  . . .  and  vill  probably  be  doing  so  for  yet 
another  year! 


Implementation 

Late  in  1988,  Pharmaceutical  Group  Sales  Training  had 
purchased  five  InfoWindow  units  (with  IBM  AT 
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microcomputers) .     These  units  formed  the  foundation  of 
the  Squibb  College  Interactive  Learning  Center.  To 
these,  Instructional  Technology  added  six  InfoWindow 
units  (with  IBM  PS/2  microcomputers) ,  one  of  which  was 
ear-marked  for  ISD  work.     Our  hopes  to  network  the 
units  have  not  yet  been  realized,  nor  have  funds  been 
available  to  purchase  units  for  other  sites. 
Programs  available  in  the  Interactive  Learning  Center 
for  independent  study  are  now  fully  employed  throughout 
Phairmaceutical  Group  Sales  Training.     These  include 
computer-based  testing  and  games,  sales  and  patient- 
management  simulations,  and  complex  videodisc-based 
instructional  programs.     In  addition,  interactive 
keypads,  interactive  games,  and  videodisc-based 
simulations  have  been  integrated  into  classroom 
training. 

Although  most  training  is  still  conducted  in  the 
traditional  manner,  we  are  pleased  that  the  integration 
of  technologies  into  training  at  Squibb  College  has  not 
stopped  with  Sales  Training.    Our  Orientation  Program 
for  New  Employees  now  encompasses  an  interactive- 
videodisc  program,  "Orientation  to  Squibb 
Cardiovascular  Products."    This  two-hour  program  is 
available  on  InfoWindow  units  that  we  have  provided  at 
five  Squibb  locations  in  addition  to  the  College 
itself.     To  accomplish  this,  we  transferred  the  five 
original  InfoWindow  units  to  other  sites,  leaving  just 
the  six  "new"  units  at  the  College. 

Our  colleagues  in  Europe  have  received  little  help  from 
us  so  far.     However,  they  are  evaluating  the 
comprehensive  computer-based  testing  program,  with  the 
intention  of  translating  it  into  French,  German,  and 
Italian.    The  adoption  or  adaptation  of  other 
individualized  programs  is  awaiting  the  availability  of 
the  new  dual-standard  systems. 


Information  Technologies  and  Future  Collage  Operations 

In  October  1989,  E.R.  Squibb  &  Sons  merged  with 
Bristol-Myers  Company  to  form  Bristol-Myers  Squibb 
Company,  the  second-largest  pharmaceutical  company  in 
the  world.    A  merger  of  this  magnitude  provides 
extraordinary  business  opportunities  for  a  corporation, 
but  also  brings  reorganization  and  redefinition  of 
roles  within  the  corporation. 
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Instructional  Technology's  Role  in  the  New  Corporation 


The  new  circumstances  under  which  we  are 
placed  call  for  new  words,  new  phrases,  and 
for  the  transfer  of  old  words  to  new  objects. 


—  Thomas  Jefferson 

As  of  this  writing  (January  1990) ,  the  role  of  Squibb 
College  (likely  to  be  called  "Bristol-Myers  Squibb 
College")  within  the  new  corporation  has  not  been 
finalized.    Undoubtedly,  we  will  retain  some  aspects  of 
our  former  mission,  but  different  areas  of 
responsibility  may  be  included.  Instructional 
Technology  will  continue  to  contribute  to  the 
achievement  of  the  College's  mission. 

We  will  likely  de-emphasize  technological  innovation 
during  the  coming  year,  in  favor  of  assessing  the 
effectiveness  of  current  practices.     During  this 
period,  we  will  have  the  time  to  achieve  benefits  from 
the  decisions  and  investments  we  have  already  made,  and 
to  make  maximum  use  of  those  productivity  and 
development  tools  we  now  possess. 


Productivity 

In  the  area  of  office  automation,  implementation  of  the 
LAN  will  allow  the  College  to  realize  the  productivity 
gains  Instructional  Technology  promised  would  come 
about  through  the  use  of  Scheduler,  e-mail,  and  shared 
files. 

We  will  fully  implement  desktop  publishing  for  the 
first  time,  producing  manuals  and  newsletters  in-house. 
And  will  produce  most  of  our  own  business  graphics  for 
overhead  transparencies  and  reports. 


Disseminating  Training 

It  will  continue  to  be  very  important  that  we 
disseminate  training  products  to  corporate  locations 
throughout  the  USA  and  Europe,  especially  to  Sales 
Training  groups.    However,  we  will  concentrate  on  less 
expensive,  more  portable  technologies  than  IBM 
InioWindow  has  proven  to  be. 

The  technologies  we  currently  favor  for  the  coming 
years  are:     audiotape,  videotape,  computer-based 
training/ testing,  and  compact-disc  interactive  systems 
(using  Digital  Video  Interactive  technology) .  With 
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these,  Instructional  Technology  can  still  achieve  our 
instructional  goals,  but  gain  levels  of  acceptance  and 
implementation^'that  eluded  us  this  past  year. 


The  seasons  have  changed 

And  the  light 

And  the  weather 

And  the  hour. 

But  it  is  the  same  land. 

And  I  begin  to  know  the  map 

And  to  get  my  bearings. 


—  Dag  Hammarskjold;  August  24th,  1961 
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Lan  Specifications: 

IBM  Token  Ring  LAN 

STF  Netware  286,  v. 2. 15 

Proteon  Intelligent  Multistation  Access  Units 
Microcom  QX/V.32C  Modem 
HP  LaserJet  IID  Printers 
HP  Color  Plotter 


l:»iisic  Desktop  System: 

IBM  PS/2  Model  50Z,  with  30  MB  hard  disk 

and  1.44  MB  3^-inch  disk  drive 
IBM  r:^il2  VGA  Monitor 
Intel  bi)2^.i  Math  Coprocessor 


Hardware  Options  (for  more  aop^i^ticated  applications) : 

IBM  PS/2  Model  80,     with  il5  MR  hard  disk 

and  1.44  MB  3^-inch  disk  arive 
IBM  8514/A  VGA  Monitor 
Intel  80387  Math  Coprocessor 
Microsoft  2-button  Mouse 
Hayes  SmartModem  2400,  with  SmartCom  II 
HP  LaserJet  II  Printer 
HP  DeskJet  Printer 


TABLE  1    College  Hardware  Systems 
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Puxrpose 

Word  processing 


Financial 
analysis 


Graphics 


Project 
management 


Communications 


Utilities 


Tools 

WordPerfect . 
v.5.1,  with 
Bitstream  fonts 
and  clip  art 


Lotus  1-2-3 . 
v.3.0 


Data  storage  and  dBASE  IV 
retrieval 


Freelance  Plus, 
v.3.0 


TimeLine ,  v. 3 . 0 


Smartcom  11, 
v.3.0 


WordPerfect 
Office 


Applications 

Correspondence 
and  documents, 
including 
macros ;  desktop 
publishing 

Budgets;  data 
display 

Vendor 

information 

system 

Overhead 
transparencies ; 
desktop 
publishing 

Project 

management ;  load 
analysis 

BBS  access; 
BITNET 

communications 

Appointment 
calendars; 
meetings 
scheduling;  e- 
mail;  simple 
databases;  file 
management ;  " to- 
do"  lists; 
calculator 


TABLE  2    Productivity  Software 
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Purpose 

Tools 

Applioations 

Student 

registration, 

administration 

Reaistrar/N. 
with  Accountant. 
Scheduler. 
Historian/N.T&D 
Analyzer 

Registration; 
delivery  costs; 
course  a   »  r^oom 
scheduli/ 
student  r  cords 

Testing 

TenCore,  Multi- 
Survey/Feud/ 
Pursuit 

Oomputer-based 
V  ;sting  and 
games 

Authoring 

IWPS ;  Summit 

Interactive 

training 

programs 

Shell 

Employes 

information 

programs 

TABLE  3    Training  Software 
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Instructional  Design  Theory: 
AdvarK«iieiTts  ftom  Cognitive  Scdenoe  and  Instructioned  Tedinology 
Scientific  advancaaents  in  cognitive  scienoe  and  instructional  technology 
suggest  significant  changes  in  lOBthods  of  curricular  and  instructicnaLL  design 
vMch  will  strongly  affect  «Jucational  practice.    Ihese  advanoeaoaents  esctend 
the  predominately  a^^lied  behaviorally-oriented  learning  paradigm  of 
instructiOTial  design  and  management.    In  this  article  I  will  discuss  three 
major  areas  in  vAiich  cognitive  science  and  instnxrtdonad  technology  are 
affecting  instructional  design  (ID)  theory.   These  are:  (a)  the  analysis  of  the 
information-to-be-leamed;  (b)  the  means  of  evalxaating  learners;  and,  (c)  the 
linkage  of  learning  theory  to  instnx±ioneQ.  prescriptions. 
Analysis  of  Infonnation"to-43e-LBamed 

An  Ijrportant  conponent  of  ID  models  is  the  anedysis  of  the  information-to- 
be-leamed.   TWo  basic  types  ot  analyses  include:  (a)  a  content  analysis,  that 
focuses  on  defining  the  critical  features  of  the  information  and  the 
relationship  of  those  features  according  to  superordinate  and  subordinate 
organissations;  and  (b)  a  task  analysis,  that  focuses  on  a  hierarchial 
organization  of  the  information  based  on  prerequisites.    Both  of  these 
analyses  identify  the  extemed  structure  of  the  information  but  do  so 
ind^)endent  of  how  it  mi^t  actually  be  stored  in  human  meimory.  However, 
researcJi  in  cognitive  psychology  on  human  memory  suggests  that  the  interrial 
organization  of  information  in  a  knowledge  base  is  based  more  on  enoplqyment 
needs  than  by  attribute  or  hierarchial  associations.    That  is,  the  utility  of 
the  knowledge  base  is  attributed  to  its  organization  not  the  amount  of 
information.    The  inplication  of  knowledge  base  organization  is  the  need  for  a 
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ftirther  analysis  of  the  information  to  better  understand  the  possible  internal 
organization  of  the  Infonnation.  Better  organizaticn  in  memory  may  also  inply 
better  accessibility  within  the  tawledge  base  for  such  higher  ortSer  cognitive 
activities  as  problem  solving  md  creativity. 

To  xmderstand  the  nature  of  kncwledge  base  organization,  cognitive 
psychologist  analysis  problem  cccplexity  and  the  way  individuals  try  to  solve 
given  problems.   By  analyzing  problems,  it  is  possible  to  identify  the 
concQjts  enplcQ^;  and,  by  analyzing  the  solutions,  it  is  possible  to  identify 
the  associations  of  those  conc^^ts  within  given  problem  situations.  The 
iirplication  for  ID  is  that  the  sequence  of  information  for  instruction  should 
be  based  in  part  on  interml  associations  as         as  external  structures.  ISie 
assunption  is  that  because  external  structures  are  independent  of  esrployment 
needs,  an  analysis  of  possible  int^emal  associations  would  inoprove  the  initial 
organization  of  the  new  information. 

In  addition  to  the  analyzing  of  problems  and  solutions,  is  the  issue  of 
problem  context.    For  exaitple,  e}q)ert  systems  reside  within  the  constraints  of 
a  specific  context:   Ihat  is,  they  can  solve  problems  only  associated  with 
that  given  context.   Likewise,  research  in  cognitive  psychology  shews  that 
individuals  can  solve  ccKplex-prc4)lems  only  if  they  poses  the  necessary 
contextual  knowledge  (i.e. ,  knowledge  of         and  why) .    For  exanple,  the 
objective  in  learning  to  play  chess  is  the  learning  of  problem  solving 
strategies  within  the  context  of  both  the  given  game  and  the  current  loove; 
not  just  hew  the  various  chess  pieces  move  (i.e. ,  procedural  knowledge) . 
UtivB,  the  key  to  both  effective  acquisition  and  enplqyment  of  knowledge  is  the 
organization  of  the  information  according  to  contextual  applications. 
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Ihe  diange  for  oontent/task  analysis  suggested  by  cxDgnltlve  science  Is  the 
method  of  infonnatlcn  analysis.   In  addition  to  the  conventional  content  and 
task  analyses,  a  context  analysis  Is  proposed  If  the  goed  of  the  Instruction 
includes  solving  canplex-problems.    Basic  st^  for  a  context  analysis  are  as 
follows: 

-Defljie  the  context  for  the  eaBployment  of  the  knowledge. 
-Define  the  carplex^-problesoos  associated  with  the  conkeoct. 
-Analysis  solutions  to  Identify  concepts  exnployed. 
•-Cluster  the  cono^rts  according  to  enployroent. 
-Organize  the  clusters  Into  an  associative  network. 

Analyzing  problesoos  within  a  context  and  then  identifying  the  concepts  vid 
their  enplqyment  organization  provides  a  means  for  sequencing  the  Instrvictian 
to  iifprove  hl^ier  order  cognition.    In  other  words,  the  sequence  of  the 
instruction  is  based  on  the  objective  of  iirproving  eitplqyinent  of  knowledge  in 
addition  to  inprovements  in  acquisition. 
Learner  Evaluation 

A  second  major  area  being  affected  by  cognitive  psychology  is  learner 
evaluation.    In  the  behavioral  paradigm,  evaliaation  focuses  only  on  observable 
student  performance  vdiile,  in  contrast,  evaluation  in  the  cognitive  paradigm 
takes  on  diagnostic  functions.   Evaluation  is  therefore  more  than  checking 
attadnment  of  bdiavioral  objectives  but,  more  inportantly,  a  concurrent  element 
of  learning.   Fbr  example,  diagnosing  learner  needs  during  instruction  has 
been  a  primary  focus  of  einplrical  research  that  itiy  colleagues  and  I  have  been 
working  on  for  the  peist  two  decades.    (A  review  of  this  program  of  research  in 
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adaptive  instruction  is  fcxmd  in  Ttennyscjn  ani  Qiristensen  [1988]  and  Tennyson 
and  Itok  [1987]). 

Much  of  the  experimental  programning  efforts  in  intelligent  occputer- 
assisted  instrrxrtion  (ICAI)  centers  on  techniques  to  identify  ndstaJoes  and 
errors  in  solving  problems  (D^  &  Schank,  1982)  •   Althou^  not  adaptive  in 
preventing  learning  problems,  ICAI  programs  are  advancing  the  concept  of 
finding  possible  problems  and  trying  to  help  the  student  by  prescribing 
specific  instruction. 

In  the  field  of  measurement,  contenporary  research  is  with  design  of 
instruments  that  measure  inference  making  within  meaningful  contexts.  Ihe 
inplication  for  instructional  design  is  the  construction  of  tests  that 
evaluation  problem  sol'^ing  within  conplex-oontext  situations:   Situations  that 
go  beyond  the  usual  limited  scope  of  measuring  only  right  and  wrong  re^^onses. 

Ihe  inplications  for  instructional  design  are  in  the  cognitive  paradigm 
concerts  of  (a)  diagnosis  of  learning  need  during  instruction  and  (b)  measures 
of  achieveiaent  within  the  context  of  meanir^ful  and  cccplex  situations.  That 
is,  can  the  learner  deal  successfully  with  the  type  of  problems  requiring 
taxwledge  of  when  and  v*iy  as  well  as  knowing  that  and  hew. 
Integrated  Instructional  Design  Model 

In  this  section  I  present  a  third  area  of  cognitive  psychology  and 
instructional  technology  influences  on  ID  theory:   Ihe  developaent  of  an 
instructional  design  model  that  focuses  on  the  planning  of  a  learning 
environment  so  that  students  not  only  acquire  knowledge  hut  also  iitprove  their 
cognitive  abilities  to  enploy  and  extend  their  knowledge.  In  Figure  1,  I 
present  an  instructional  design  model  that  shows  the  direct  integration  of 
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cognitive  laming  theory  with  prescribed  ir^tructicaial  strategies.  In 
addition  to  the  direct  lixikages  between  the  various  inemory  systesns  and 
instructional  strategies,  I  propose  that  70%  of  formal,  classroom  learning 
tiine  use  instriKtional  prescriptions  that  focus  on  eaooploying  and  inproving 
kno/ledge.   The  major  components  of  the  ID  model  are:  memory  systans,  learning 
objectives,  Instructional  time,  and  instructionad  prescriptions*   These  are  new 
discussed  in  turn. 


Insert  Figure  1  about  here 


Memory  systems^    The  proposed  ID  model  is  directly  associated  to  the 
cognitive  paradigm  of  learning.   A  cocplete  discussion  of  this  paradigm  is 
fould  in  Tennyson  and  Basch  (1988)  •    Because  the  goals  of  the  ID  model  include 
both  the  acquisition  and  the  espplqym^  of  laxwledge,  the  mesnory  systems 
reference  the  long-term  memory  subsystems  of  storage  and  retrieval.  The 
storage  system  is  composed  of  three  basic  forms  of  knowledge:  Declarative 
knowledge,  knowing  "that"  about  the  infonnation;  procedural  knowledge,  knowing 
••x'low''  to  use  information;  and,  contextual  taKWledge,  knowing  •»v*ien  and  \Ary*»  to 
use  given  information.   The  retrieval  system  is  ccnposed  of  cognitive  abilities 
associated  with  the  processes  of  recall,  problem  solving,  and  creativity. 

Proposed  in  the  ID  model  (see  Figure  1)  is  that  there  is  a  connection 
between  the  five  basic  memory  systems  and  prescribed  instructional  strategies. 
The  purpose  for  indvding  this  oowponent  in  the  ID  model  is  twofold:   First,  to 
establish  a  direct  link  between  instructional  ttieory  and  learning  theory:  This 
was  done  successfully  with  the  behavioral  paradigm  ^diere  instructional 
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strategies  were  designed  follcwing  tlie  cxaiditions  of  that  paradigm.   Ihus,  I 
have  attenpted  in  this  article  to  make  an  association  bet^^den  the  cognitive 
paradigm  and  instructional  strategies.   And,  second,  to  jjdicate  the  relative 
strengths  of  the  instructional  strategies  in  reference  to  the  educational 
goals  of  knowledge  acquisition  and  enplojyment.   The  focus  of  the  cognitive 
paradigm  is  on  both  of  these  educational  goals  as  contrasted  to  the  b^iavdoral 
paradigm  vihich  only  conoeMis  itself  with  the  acquisition  goal.   Within  the 
proposed  ID  nodel,  the  learning  objectives  extend  the  macro  level  goals  so  as 
to  tie  directly  with  the  manory  systems  cacponent. 

learning  objectives.   Ihe  purpose  of  oognitive-taised  learning  objectives  is 
to  further  elaborate  the  curricular  goals  of  knowledge  acquisition  and 
enplqyinent.   Objectives  are  iirportant  in  the  planning  of  learning  environments 
becaxase  they  provide  the  loeans  of  both  allocating  instnactional  tiine  and 
identifying  ^5ecific  instructional  strategies.    I  define  learning  objectives 
as  follows; 

-Verbal  informaticai.   This  objective  deals  with  the  learner  acquiring  an 
awareness  and  understanding  of  the  cono^3ts,  rules,  and  priixsiples  within 
a  specified  domain  of  infomation  (i.e.,  declarative  knowledge). 

-Intellectual  skills.   Ihis  objective  involves  the  learner  acquiring  the 
skill  to  correctly  use  the  concepts,  rules,  and  principles  of  a  specified 
dcKain  of  jnfonnation  (i.e.,  procedured  knowledge). 

-Oontextual  Skills.   This  objective  focuses  on  the  learner's 
acquisition  of  a  knowledge  base's  organization  and  accessibility  (i.e., 
contextual  knowledge) .   The  organization  of  a  knowledge  base  refers  to 
the  scheanatic  structure  of  the  information  whereas  the  accessibility 
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refers  to  the  executive  control  strategies  that  provide  the  means 
necessary  to  enplcy  the  knowledge  base  in  the  service  of  recall,  problem 
solving,  and  creativity.   CSontextual  knowledge  li^udes  the  criteria, 
valiaes,  and  2$)propriateness  of  a  given  detains  schematic  structure.  For 
example,  sJuDoply  kncwing  how  to  classify  examples  or  kncvdng  how  to  use  a 
rule  (or  principle)  does  not  iaply  that  the  learner  knows  \*ien  and  v4iy  to 
einplcy  specific  comsg/ts  or  rules. 

-Ocxpiitive  strategies.    This  objective  deals  with  both  the  developoaent  of 
cognitive  oonplexity  abilities  and  the  iicprovement  of  domain  specific 
strategies  of  thinking.   Thus,  this  category  of  learning  objectives  deals 
with  two  lirportant  issu^  In  education.   First,  the  elaboration  of  thinking 
strategies  that  will  ana  the  students  with  increased  docain  specific 
contextual  knowledge.   Second,  the  development  of  the  cognitive  abilities 
of  differentiation  and  integration.    Ihese  abilities  provide  the  cognitive 
tools  to  effectively  enplqy  and  iiiprove  the  knowledge  base;  therefore,  they 
are  integral  to  any  educaticxial  goal  seeking  to  inprove  thinking 
strategies. 

-Creative  Processes.   This  objective  deals  with  the  most  elvjsive  goal  of 
education:  the  develcpnent  and  improvement  of  learner  creative  abilities. 
I  define  creativity  as  a  two  fold  ability:   First,  creating  knowledge 
to  solve  a  prcblem  froa  the  external  environment;  and,  seKsond,  creating 
the  problem  as  well  as  the  knowledge.    Integral  to  the  creating  of  both 
the  problem  and  kncwledge  is  the  criteria  by  which  consistent  judgeonaent 
can  be  made.    I  define  two  forms  of  criteria  as  follcws:   The  first  is 
criteria  that  are  known  and  v*iich  can  be  applied  with  a  hi^  level  of 
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cxnsistenqy.    In  contrast  are  criteria  that  are  developed  cxaxairrently 
with  the  problem  wid/or  knowledge,  and  is  csonsistently  applied  across  a 
hi^  level  of  pmJuctivity.    Cceativi'b^  objectives  need  to  specify  not 
only  the  ability  to  develope  and  inprove,  but  also  the  fona  of 
criteria.   Hxat  is,  students  should  be  informed  of  the  criteria  in  the 
former  and,  in  the  latter,  the  necessity  to  develop  criteria. 
Insxxuctional  tijne.   A  key  factor  in  litplementlng  the  cognitive  goeds  of 
knov/ledge  acquisition  and  enplqyment  is  the  allocation  of  learning  time  by 
defined  objectives.  For  exanple,  Tennyson  and  Rasch  (1988)  suggest  that  if 
inprovements  in  problem  solving  and  creativity  are  to  occur,  there  needs  to  be 
a  significant  change  in  hew  instructional  time  is  allocated.    They  reocmnend 
that  the  conventional  instructional  time  allocation  for  learning  be  altered  so 
that,  instead  of  70%  of  instnaction  aimed  at  the  declarative  and  procedural 
knowledge  levels  of  learning,  70%  be  devoted  to  learning  and  thinking 
situations  that  involve  acquisition  of  contextual  knowledge  and  development  of 
cognitive  abilities. 

Using  Tennyson  and  Rasch's  recommended  figures  on  instrxocticaial  time 
allocation,  I  propose  that  learning  time  be  divided  between  the  two  main 
subsystems  of  long-time  memory — storage  and  retrieval.   Within  the  guidelines 
illustrated  in  Figure  1,  time  is  assigned  according  to  the  cognitive  objectives 
defined  in  the  previous  section.   In  the  storage  system,  learning  time  is 
allocated  among  the  three  memory  systems  making  \jp  a  knowledge  base  as 
follows:  declarative  knowledge  10%;  procedure  knowledge  20%;  and  contextual 
knowledge  25%.    I  am  recommending  that  contextual  knowledge  learning  time  be 
about  equal  to  the  other  two  knowledge  forms  because  of  the  necessity  to  both 
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organize  a  knowledge  base  and  cSevelop  accessibility.   Uie  value  of  a  knowledge 
base  is  primarily  in  the  functionality  of  its  organization  and  accessibility  . 
Without  a  sufficient  base  of  contextual  knowlec^,  the  opportunity  for 
enplqyioent,  future  elaborations,  and  extension  of  the  knowledge  base  is 
severely  limited. 

For  the  knowledge  acquisition  goal,  the  focus  of  leaniing  time  allocation 
is  on  ccarrtextual  knowledge,  and  away  from  the  xisual  practice  of  heavy  einphasis 
on  amount  of  inforr^tion.    I  am  assuming  that  declarative  and  procedural 
knowledge  acquisition  is  an  interactive  process  that  is  improved  v*yen  eaoplcying 
the  knowledge  base  in  the  service  of  hi^ier-order  thinking  situations  (i.e., 
problem  solving  and  creativity) .   Time  allocated  for  declarative  and  procedural 
knowledge  focuses  on  establishing  an  initJ^  base  of  necessary  knowledge  that 
can  be  used  within  a  context  of  a  prctolem  situation.   That  is,  learning  time 
should  incaxade  the  opportunity  for  the  learner  to  gain  experience  in  enploying 
the  knowledge. 

The  learning  times  presented  in  Figure  1  do  not  iirply  a  linear  sequence  of 
knowledge  acquisition  going  fron  declarative  to  contextml.   Rather,  they 
r^resent  total  amounts  in  an  iterative  learning  environment  v*iere  learners 
are  continuously  acquiring  each  form  of  knowledge.    For  exanple,  students  may 
engage  in  contextual  knowledge  acquisition  prior  to  decdarative  knowledge 
acquisition  if  they  currently  have  sufficient  background  kncwledge  (i.e.,  a 
discovery  method  of  instuiaction  as  contrasted  to  a  sx:nx±ured  method) . 

Instructional  prescripticais,   Hie  purpose  of  the  proposed  ID  lODdel  is  the 
direct  linkage  of  instructional  strategies  to  specific  memory  system 
ocaooponents.  Also,  instead  of  prescribing  a  given  st:rat::egy  of  instruction  for 
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all  form  of  learning,  I  havB  identified  general  categories  of  strategies, 
each  occpcsed  of  variables  and  (xnditions  that  can  be  manipulated  according  to 
given  instructicsial  situations. 

Ihe  five  instructional  strategy  categories  are  as  follows: 

Expository  strategies.   This  category  represents  those  instructional 
variables  designed  to  provide  an  environment  for  learning  of  declarative 
loKPWledge  (see  Figure  1) .   The  basic  jjistnjctional  variables  provide  a  context 
for  the  to-be-learned  information.    That  is,  the  cono^  of  advance  organizers 
is  extended  by  presenting  a  meaningful  context  for  the  information  as  well  as 
a  mental  framework  of  the  given  danains  abstract  stmcture.    In  addition  to 
providing  a  context  for  the  information,  meaning  can  be  farther  enhanced  by 
adapting  the  context  to  individual  student  badcground  knowledge  (Ross,  1983)  • 

Tt)B  context  establishes  not  only  the  initial  organization  of  the  dcnain 
but,  2dso,  introducjes  both  the  '^*y"  of  the  theoretical  nature  of  the 
information  and  the  '*vdien"  of  the  criterion  nature  of  the  dciDains  standards, 
valiaes,  and  appropriateness.   Person\Lizlng  the  context  to  student  background 
knowledge  linproves  understanding  of  the  information  by  connecting,  within 
working  memory,  knowledge  that  is  easily  retrieved.    Ihios,  the  new  Knowledge 
becomes  directly  linked  or  associated  with  existing  schemas. 

Following  the  ocaitextual  Introduction  of  the  information,  additional 
&cpL    zary  instructional  variables  present  the  ideas,  cono^rt:s,  principles, 
rules,  facts,  etc.  in  forms  that  extend  existing  knowledge  and  that  aid  in 
establishing  new  ]<ncwledge.   These  variables  include: 

-Label,   Althcaig(h  a  sluple  variable,  it  is  often  necessary  to  elaborate 
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on  a  label's  origin  so  that  the  student  is  just  not  trying  to  nenorize  a 
nonsense  word. 

-Definition.   liie  purpr<  a  of  a  definition  is  to  link  \jp  the  new 
information  with  existing  knowledge  in  long-term  memory;  otherwise  the 
definition  may  convey  no  meaning,    niat  is,  the  student  should  know  the 
critical  attributes  of  the  oonoept.   To  farther  Improve  imderstanding  of 
the  new  information,  definitions  may,  in  addition  to  presentation  of  the 
critical  attributes  (i.e.,  prerequisite  kna;ledge)  iixd.ude  infoniBtion 
linked  to  the  student's  background  knowledge. 

-Best  Exaitple.    To  help  students  establish  cleax'  abstracts  of  a  domain's 
concqpts,  an  initial  exanple  should  represent  an.  easy  occprehension  of 
the  given  concert:  (or  rule,  principle,  idea,  etc.).  Additional 
ejqxDsitory  exaitples  will  enhance  the  depth  of  understanding. 

-E>5)ository  Exanples.   Additional  exsjiples  should  provide  increasingly 
divergent  applications  of  the  infonnation;  perfis?©  also  in  alternative 
contexts) . 

-Worked  Exanples.   Ihis  variable  provides  an  expository  environitent  in 

the  information  is  presented  to  the  student  in  statement  forms  that 
elaborate  application*   Bie  purpose  is  to  help  the  student  in  beoaa-^-^ 
aware  of  the  afplication  of  the  information  within  the  given  context(s) . 
Jtar  exanple,  to  learn  a  mathematical  operation,  the  student  can  be 
presented  the  steps  of  the  process  in  an  e^qposLtary  problem  vMle, 
concurrently,  presenting  e55)lanations  for  each  st^.    In  this  way,  the 
student  may  more  clearly  understand  the  procediares  of  the  mathematical 
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operation  without  developing  possible  miscxaKseptions  or 
overgeneralizatioDs . 

Kractice  stratecdes,   This  category  of  instructional  prescriptions 
contains  a  rich  variety  of  variables  and  conditions  vAiidi  can  be  designed  intx) 
nuineroias  strategies  to  iirprove  learning  of  procedural  knowledge.  iSiis 
category  is  labelled  practice,  becaxase  the  objective  ir.  to  learn  hov  to  use 
knowledge  correctly;  therefore,  it  requires  constant  interaction  between 
student  learning  (e.g. ,  problem  solving)  and  instructional  system  xnonitoring. 
Practice  strategies  should  atteocpt  to  create  an  enviroransnt  in  vSiich  (a)  the 
stident  learns  to  apply  kncwledga  to  previously  unenoountered  situations  vftiile 
(b)  the  instructional  system  carefully  monitors  the  student's  performance  so 
as  to  both  prevent  and  correct  possible  miscono^Jtions  of  procedural  knowledge. 

Hie  basic  instructional  variable  in  this  strategy  is  the  presentation  of 
problems  that  have  not  been  previously  encountered  (see  Tennyson  & 
Oocciiiarella,  1986,  for  a  oonplete  review  of  variables  in  this  category) . 
Other  variables  inclide  Lieans  for  evaluation  of  learner  responses  (e.g. , 
pattern  recognition) ,  aidviseroent  (or  coaching) ,  elaboration  of  basic 
information  (e.g.,  text  density,  Morrison  et  al.,  1988),  formal:  of  information, 
number  of  problems,  use  of  expo^^tory  inforration,  error  analysis,  and  lastly, 
refreshment  and  remediation  of  prerequisite  information. 

Pro>?^^i-orlented  strategies.    In  the  proposed  ID  model  (Figure  1) ,  I 
propose  that  25%  of  the  instructional  time  be  allocated  to  the  acquisition  of 
contextual  knowledge.   A  proposed  instr\:fectional  strategy  for  this  category 
uses  problem-<)riented  simulation  techniques.   The  purpose  of  s"*TnulatiQns  is  to 
iirprove  the  organf,zation  and  accessibility  of  information  within  a  knowledge 
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base  by  presenting  problems  that  require  the  student  to  search  thrtx^  their 
itvemory  to  locate  and  retrieve  the  appropriate  kncwledge  to  propose  a  soliiticxi. 
Within  this  context,  the  simulation  is  a  problem  rather  than  an  expository 
demonstration  of  sone  situation  or  phencmencn. 

ProblesQEt-oriented  simulations  present  dcnain  ^)eci£ic  problesn  situations  to 
inprove  the  organization  and  accessibility  of  information  within  the  taxwledge 
base.   Basically,  the  strategy  focuses  on  the  stiiflents  trying  to  \:ise  their 
declarative  and  procedural  knowledge  in  solving  danaain-epecific  problews. 
Problem-oriented  simulations  present  problem  situations  that  require  the 
student  to  (a)  analyze  the  problem,  (b)  vrork  cut  a  oonoeptuedization  of  the 
problem,  (c)  define  specific  goals  for  coping  with  the  problem,  and  (d) 
propose  a  solvrtion  or  decision.   Unlike  problems  in  the  practice  strategies 
that  focus  on  acquiring  procedural  krcwledge,  problem-oriented  simulations 
present  situations  that  require  employment  of  the  doraain's  procedural 
knowledge.   Ihus,  the  student  is  in  a  problem  solving  situation  that  requires 
establishing  connections  and  associations  among  the  facts,  conc^jts,  rules, 
and  principles  of  specific  dcnains  of  information. 

Conplex-dynamic  stratecfies.   Enployment  of  the  kncwledge  base  in  the 
service  of  recall,  problem  solving  and  creativity  is  the  second  major 
educational  goal  of  learning  environments.    In  contrast  to  instructional 
systems  that  vise  danain-ind^)endent  thinking  skills  development,  this 
instriKtional  strategy  category  proposes  the  presentation  of  docain-specific 
situations  that  allow  learners  to  develop  their  thinking  processes  vMle 
enplqying  the  dotain  kncwledge  stored  in  r^eir  own  inemory  systems.   As  such, 
ccwplex-K^imamic  similations  extend  the  format  of  the  problem-oriented 
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simulations  by  use  of  an  iterative  fonnat  that  not  only  shows  the  consequences 
of  decisions  but  also  updates  the  situational  conditions  and  proceeds  to  sake 
the  next  iteration  more  ocnplex.   That  is,  the  situation  is  longitudinal, 
allowing  for  increasing  difficulty  as  well  as  providii^  additions,  deletions, 
and  changes  in  variables  and  conditions.    In  sophisticated  conplex-^Jynamic 
simulations  these  alterations  and  changes  are  done  according  to  individual 
differences  (Breuer,  1987). 

ihe  main  features  of  ccnplex-dynamic  siioulations  are:    (a)  present  the 
initial  variables  and  conditions  of  the  situation;  (b)  assess  the  learner  's 
proposed  solution;  and  (c)  establish  the  next  iteration  of  the  variables  and 
conditions  based  on  the  cumulative  efforts  of  the  learner  (Breuer  &  Ruraner,  in 
press). 

In  summary,  ccoplex-dynamic  strategies  should  be  designed  to  provide  a 
learning  environment  in  which  learners  develop  and  inprove  higher-order 
thinking  processes  by  engaging  in  situations  that  require  the  employment  of 
their  knowledge  base  in  the  service  of  prx±»lem  solving. 

Self -directed  experiences.   Bie  ca?eative  process  is  a  cognitive  ability 
that  seemingly  can  be  iaproved  by  learners  who  engage  in  activities  requiring 
novel  and  valuable  cutccmes.   Ihat  is,  the  creative  proceas  can  be  irtproved  by 
instructional  methods  that  allow  students  the  opportunity  to  create  knowledge 
within  the  context  of  a  given  domain.    Instructional  programs  that  provide  an 
environment  for  easy  manipulation  of  new  information  increase  the  learning 
time  available  for  such  activities. 

Oonputer-based  software  programs  provide  environroents  for  self-directed 
learning  ©^jerienocs  that  may  inprove  the  creative  p^joess  within  given 
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dcraains.    For  exaitple,  word  processing  programs  have  been  shewn  to  iirprove 
vilting  skills  because  of  the  ease  in  correcting  and  adjiisting  text  structure 
(Lawler,  1985;  Zvacek,  1988)  •    Ocnputer-^^ased  sinulations  have  also  shewn  that 
the  creative  process  can  be  inproved        stixients  can  both  continually  see 
the  outccmes  of  their  decisions  v*iile  imderstanding  the  predictability  of 
their  decisions  (Sasch,  1988} . 

The  creative  process  is  a  cognitive  ability  that  aqpparently  can  be 
iitproved  with  xose  within  a  daooain  and  oaoaputerH^ased  software  ;  ograms  seem  to 
pixjvide  the  t^^pe  of  environment  Vihidi  can  enhance  instructional  laethods  for 
such  inproveiaents.    Because  of  the  tiioe  necessary  for  participating  in  creative 
activities,  educatoi-s  should  provide  sufficient  learning  tirae  for  such 
developoaent  (Tennyson,  1988) .    Conputer  software  progracras  that  are  dcssain 
specific  enhance  the  cost-effectiveness  of  instrtjctional  strategies  aiioed  at 
the  liiproveinent  of  creativity. 

The  key  instrtictional  attribute  for  this  category  is  an  environment  that 
allows  students  to  e)5)eriexK5e  the  creative  process  at  that  given  moment. 
Ocarputer  software  programs  that  are  donain  specific  and  provide  for  self- 
directed  leamii^  seem  to  offer  excellent  instructional  strategies  for  loeeting 
goals  of  a  curriculum  that  emphasizes  higjier-level  thinking  strategies. 
Althou^,  we  have  focused  on  oai^uter-based  software  in  this  instructional 
category,  there  are  of  course  other  possible  instructiOTial  ineans  for  students 
inproving  their  creative  processes. 

(Conclusion 

The  purpc^  of  this  article  was  to  discuss  several  areas  in  vdiidi  recent 
advancements  in  cognitive  psychology  and  educationa].  technology  may  affect 
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instnictional  design  theory.   The  first  two  areas,  information  anadysis  and 
learner  evalmtion,  proposed  extensions  to  current  methods  of  instnictional 
development*   Information  analysis  proposes  an  additional  analysis  of  the 
information  based  i^xai  cccplex-problems  associated  with  a  given  oontext* 
Vftiereas,  a  conventional  contenVtask  analysis  identify  the  attrllxctes  of  the 
information,  the  context  analysis  identifies  the  scheaoKitic  organization  of  the 
information.   Hie  sdienatic  organization  luproves  the  service  of  the  knowledge 
base  for  hic^ier  level  enplqyraent  situations  (i.e.,  probleaoa  solving  and 
creativity) . 

The  effect  of  cognitive  psychology  on  learner  evaluation  deals  with  two 
fundamental  aspects  of  evalixation.    First,  when  do  it  and,  second,  vAiat  to 
measure.    For  the  former  aspect  of  v*ien,  it  is  proposed  that  evaluaticxi  be  of 
a  diagnostic  nature,  occurring  during  learning.    Diagnosing  learning  need 
\Mle  leamii^  would  iajpzove  the  instructional  prescription.   Ihat  is,  the 
instructional  program  would  be  adjusting  to  current  individual  needs  and 
differences.   For  the  latter  aspect,  the  focus  of  evalxoation  would  be  on 
hitler  order  inference  making  both  within  specific  doitains  and  cognitive 
abilities. 

The  third  area  prc^^sed  the  framework  for  an  ID  model  that:  directly  links 
cognitive  learning  theory  with  specific  instructional  strategies.    I  also 
proposed  that  learning  time  be  allocated  acnording  to  the  educational  goals  of 
knowledge  acquisiticxi  and  errployment.   Given  the  assuraption  that  learners  can 
elaborate  and  extend  their  knowledge  base  during  enployment,  instructional 
time  was  shifted  from  the  traditional  allocation  of  70%  for  acquisition  to  70% 
for  iitprovements  in  enployment.    This  reallocation  of  instructional  time  does 
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not  reduce  the  iicportanoe  of  cxDntent,  but  places  more  of  the  burden  of 
acquisition  on  the  learner.   Rather  than  acquisition  of  kncwledge  in  nonsense 
isolation,  it  is  proposed  that  learners  acquire  kncwledge  within  meaningful 
situations.   Unfortunately,  research  in  instructional  thecxty  has  focused  on 
strategies  associated  with  declarative  and  procedural  kncwledge  witli  minimal 
eitpirical  work  for  strategies  associated  with  contextual  kncwledge  and 
cognitive  abilities.   Educational  technology  should  provide  the  means  by  whidi 
cognitive  psychology  can  be  applied  to  inprovements  in  learning.   5hat  is,  the 
behavioral  paradigm  was  litplemented  b^^  means  of  educational  technology,  I  see 
the  same  thing  happening  for  cognitive  psychology. 
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Figure  Cs^jtions 

Fic^oTD  1.    InstnactiOTal  design  itcdel  linking  cxgnitive  learning  theory  with 
instructional  prescriptions. 
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Recent  advances  in  computers  have  provided  capabilities  to  develop 
sophisticated  simulators  for  training  and  embedding  the  training  within  operational 
equipment.  Embedded  training  Is  an  emerging  trend  In  military  education.  The 
use  of  simulators  and  embedded  training  (ET)  has  Increased  for  training  of  pilots, 
weapon  systems,  etc.  Some  of  the  reasons  for  this  are  cost,  safety,  practice  time, 
and  the  ability  to  train  in  adverse  conditions  at  any  time.  The  effectiveness  of 
simulators  and  ET  is  measured  by  level  of  fidelity,  cost,  feedback,  transfer  of 
training  to  the  operational  equipment,  and  Increase  in  skill  level.  This  paper 
reports  an  investigation  into  the  direction  that  embedded  training  is  taking  and 
examines  lessons  learned  about  the  when  and  how  to  Implement  embedded 
training. 

The  purpose  of  training  devices  and  embedded  training  Is  twofold:  first,  to 
provide  opportunities  for  the  student  to  practice  his/her  job  performance,  and; 
secondly,  to  provide  original  learning  opportunities  for  the  students.  The  term 
"Embedded  Training"  (ET)  has  been  defined  as  training  that  Is  provided  by 
capabilities  designed  to  be  built  into  or  added  onto  operational  systems  to 
enhance  and  maintain  the  skill  proficiency  necessary  to  operate  and  maintain  that 
equipment  end  Item  (1:4-8). 

ET  has  several  advantages  over  the  traditional  approaches  of  either 
supervised  OJT  (on-the-job  training)  or  CB"  (computer-based  training)  using 
stand-alone  personal  computers  (PCs).  First,  embedded  training  allows  Immediate 
hands-on  experience  with  the  operational  system.  Second,  ET  provides  self-paced 
practice  in  critical,  hard-to-master  skills  without  the  man-lntensiveness  of  over- 
the-shoulder  tutoring.  Finally,  by  stimulating  actual  workstation  equipment,  the 


1 


ERIC 


63^ 

632 


embedded  training  component  can  provide  the  learner  with  realistic,  system- 
specific  cues,  responses,  and  Information.  Additionally,  since  H  can  use  actual 
operational  system  data,  it  can  provide  a  mechanism  for  sidll  qualification  testing 
that  is  site-specific  or  operational-specific  rather  than  generic. 

Several  types  of  tasks  are  better  candidates  for  training  via  ET.  These  are 
summarized  in  Figure  1.  Generally  it  is  more  i»ppropriate  to  training  tasits  via  ET 
which  require  higher  level  cognitive  sicills,  the  Integrated  performance  of  muitipple 
skills,  or  team  training.  Tasks  which  are  primarily  rote  actions  or  which  require 
only  lower  levels  of  cognitive  activity  are  poorer  candiaates  for  inclusion  in  ET. 


GOOD 

EMBEDDED  TRAINING 
CANDIDATES 


POOR 

EMBEDDED  TRAINING 
CANDIDATES 


•  TEAM  COOPERATION  TASKS  •  ROTE  PRECEDURES 

•  UTILIZATION  OF  RULES  OR  CONCEPTS  •  BASIC  KNOWLEDGE 

.  PERF0RM;^-CE  of  VARIABLE  OR  '  ^^^^  ^^^^^  BEHAVIORS/SKILLS 

CONTINGENCY  PROCEDURES 

■  PERFORMANCE  OF  INTEGRATED 
MULTIPLE  SKILLS 


Rgure  1.  Task/Embedded  Training  Suitability 


The  basic  technology  for  ET  is  computer-based.  Any  control  system  which 
is  digital  in  format  and  can  provide  sufficient  memory,  processing,  and  storage  to 
handle  the  training  database  is  a  candidate  for  embedded  training.  ET  concepts 
are  still  in  an  infancy  stage  when  compared  to  advances  in  technology  which 
might  be  utilized  for  ET.  While  ET  has  been  utilized  by  the  military  on  isolated 
projects  for  over  30  years,  only  recently  has  direction  been  given  to  thoroughly 
evaluate  and  consider  ET  as  the  preferred  training  alternative.  ET  is  now  specified 
as  the  preferred  training  alternative  for  the  design  &  development  of  training 
subsystems  in  the  development  and  foliow-on  Product  Improvement  Programs  of 
all  Army  materiel  systems  by  the  Joint  Chiefs  of  Staff  (JCS). 
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Several  types  of  training  objectives  have  been  identified  which  more 
appropriately  lend  themselves  to  an  Embedded  Training  approach.  Generally 
tasks  car)  be  identified  as  appropriate  for  ET  based  on  their  criiicaiity  and 
perishability  (2: 17ff).  Criticality  is  a  vaiuative  judgement  on  the  consequences  on 
inadequate  or  slow  task  performance.  Perishability  Is  a  rating  Indicating  the  skill 
decay  rate  of  a  component  skill  including  decay  of  the  ability  to  perform  tasks  or 
objectives  which  are  dependent  on  that  skill.  Generally  tasks  which  require  the 
performance  of  integrated  multiple-skills  (high-workload,  demanding  components 
of  performance),  the  utilization  of  rules  or  concepts  (complex  ccgnitlve  tasks),  and 
the  performance  of  variable  or  contingency  procedures  (raquiring  flexible, 
mediated  responses)  are  more  suitable  for  Embedded  Training.  Figure  2 
summarizes  primary  ET  task  selection  factors.  Tasks  which  involve  rote  or 
invariate  procedures,  basic  manipulative  skills,  factual  knowledge,  and  basic  level 
behavior^  are  usuaity  less  suitable  for  Inclusion  In  embedded  training.  However, 
extenuating  circumstances  may  justify  their  inclusion  for  ET.  Embedded  training 
is  now  recognized  as  a  viable  alternative  for  consideration  when  degradation  in 
skill  proficiency  is  likely  due  to  infrequent  application  or  where  constant  practice 
is  needed  to  maintain  performance  proficiency. 


•  Criticality 

•  Perishability 

•  Consequences  of  Improper  Performance 

•  Hands-on  Training  Needed? 

•  Delay  Factor 

•  Complexity  of  Task 

•  Safety  Factors 

•  Fidelity  Requirements 


Figure  2.  Embedded  Training  Task  Selection  Factors 

ET  was  first  successfully  employed  In  the  50s  with  the  development  of  the 
Semi  Automatic  Ground  Environment  (SAGE)  System  for  the  U.S.  Air  Force.  While 
this  embedded  training  application  was  successful,  there  was  little  movement  to 
consider  Incorporation  ET  into  other  systems  (3:515). 

During  the  60s,  embedded  training  was  used  for  the  Pershing  missiie 
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systems,  T1  SImulatoj  for  the  Nike  Hercules,  the  AN/TPQ-21  Basis  Hawk  Missile 
System,  the  Automated  Programmed  instruction  (API)  and  Astronaui  training  for 
NASA  During  the  70s,  ET  was  used  for  the  upgrade  of  the  Basic  Hawk  Missile 
System  to  the  ANyTPQ-29  Improved  Hawk  Missile  System.  The  Programming 
Language  for  Interactive  Teaching  (PLANIT)  was  also  tested  and  validated  in  ET 
applications  for  teaching  non-system-tied  military  f,k\\ls  during  the  MASSTER  122 
test  and  for  teaching  tactical  data  system  skills  using  the  Tactical  Fire  Direction 
System  (TACFIRE). 

During  the  80s,  the  application  of  ET  has  increased.  Embedded  Training  was 
the  training  concept  for  the  Fiber-Optic  Guided  Missile  (FOG-M),  the  Army's 
Howitzer  Improvement  Program  (HIP),  and  the  SGT  York.  The  Navy  used  ET  on 
the  Language  Translator  (L-TRAN)  for  the  Naval  Tactical  Data  System  (NTDS),  and 
AN/SPA-25G  Radar  Repeater,  the  On-Board  Maintenance  Trainer  (OBMT)  and  the 
AEGIS  Combat  Training  System  (ACTS)  to  name  a  few.  Also,  the  Air  Force's  On- 
Board  Simulation  (CBS)  for  the  F-15,  On-Board  Electronic  Warfare  Simulator 
(OBEWS)  and  Pilot  training  uses  ET. 

Some  of  the  ET  programs  currently  under  development  for  the  Army  are  the 
following:  Guard  Unit  Armory  Device  (GUARD)  Full-Crew  Interactive  Simulator 
Trainer  (FIST)  -  Armor  (GUARDFIST-I),  Ught  Helicopter  (LHX)  Family,  Advanced 
Antitank  Weapon  System  -  Medium  (AAWS-M),  Multiple  integrated  Laser 
Engagement  System  (MILES)  -  Air  to  Air  Stinger  (ATAS)(MILES-ATAS),  the 
Armored  Family  of  Vehicle  (AFV)  Program,  and  the  Fon  -nrd  Area  Air  Defense 
System  (FAADS).  The  Air  Force  Is  developing  the  use  of  £T  for  Aircraft  cockpit 
training.  High  Speed  Flyouts,  and  Reconnaissance  Attack  Fighter  Training  System 
(RAFTS)  a  T-38  replacement. 

Army  doctrine  requires  that  identification  of  the  training  device  needs  to 
occur  early-on  during  the  formulation  of  the  Required  Operational  Capability 
(ROC).  During  the  development  of  the  ROC,  ET  is  considered  and  must  be 
identified  as  a  requirement,  if  applicable,  in  the  operational  characteristics  and 
training  assessment  paragraphs.  Identifying  the  requirement  foi  ET  as  an 
essential  characteristic  of  the  system  causes  ET  to  be  developed  as  part  of  the 
operational  system.  ET  issues  must  be  considered  early  enough  to  be 
incorporated  into  initial  prototype  designs.  ET  capabilities  to  train  individual  tasks 
through  force-level  collective  tasks  has  been  described  In  AR  350-38.  Embedded 
training  is  described  as  encompassing  four  training  categories: 

(1)  Category  A  -  individual/operator  training  objective:  To 

attain  and  sustain  individual,  maintenance  and  system  orientation 
skills. 

(2)  Category  B  -  Crew  training  objective:  To  sustain  combat 

ready  crews/teams.  This  category  builds  upon  skills  acquired  from 
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Category  A. 

(3)  Category  C  •  Functional  training  objective:  To  train  or 

sustain  cornmanders,  stans,  and  crews/teams  within  sach  functionai 
area  to  be  utilized  In  their  operational  role. 

(4)  Category  D  •  Force  Level  (Combined  Arms  Command  and  Battle  Staff) 

training  objective:  To  train  or  sustain  combat  ready  commanders  and 
battle  staffs  utilizing  the  operational  system  in  its  combat  operational 
role. 

The  determination  of  appropriate  category  application  for  a  given  type  of 
system/training  strategy  has  not  been  established,  nor  have  any  heuristics  for 
applying  ET  to  meet  the  objectives  of  a  given  category  been  documented. 
Training  developed  follows  AR  350-38,  Training:  Training  Device  Policies  and 
Management,  as  a  guide  to  define  the  term  Embedded  Training  (ET).  Through  a 
thorough  analysis  using  ARI  80-15  "How  to  Determine  Training  Device 
Requirements"  and  the  new  ARI  ten  volume  Series:  Implementing  Embedded 
Training,  the  training  developer  determines  the  training  need  and  requirements. 
In  conjunction  v.'ith  this  effort  a  methodology  to  determine  the  stimulus 
requirements  and  levels  of  fidelity  needed  for  effective  training  is  sometimes 
conducted  based  upon  The  Army  Evaluation  Group  (TAEG)  and  contained  in 
Reports  No.  16  and  No.  23,  Focus  on  the  Trained  Man.  These  efforts  are  tailored 
to  the  process  of  specifying  ET  characteristics  to  achieve  the  most  effective 
instructional  delivery  system. 

Generally  ET  progresses  through  three  major  analysis  stages.  These  stages 
are  illustrated  in  Figure  3.  During  initial  concept  analysis  an  examination  of  the 
hardware  and  software  constraints  of  the  actual  system  must  be  conducted. 
Additionally  tasks  and  human  performance  requirements  for  the  system  must  be 
investigated.  During  the  second  phase,  the  information  gathered  during  concept 
analysis  is  used  to  develop  conditions  and  standards  for  task  performance,  task 
practice  requirements,  and  to  establish  training  priorities  for  ET.  During  the  third 
phase  of  ET  analysis  the  ET  training  component  of  the  system  is  defined,  and  the 
detailed  ET  design  is  developed.  This  entire  process  is  interative  and  must  be 
adjusted  to  changes  made  in  the  developing  system.  Often  the  real  system  is 
limited  in  its  capacity  to  handle  the  entire  ET  requirements.  The  ET  component 
design  must  be  maximized  within  the  limits  of  the  system. 

Once  the  system  design  and  the  ET  design  are  known  and  are  relatively 
stable,  development  of  the  ET  component  must  begin.  This  process  is 
summarized  in  Figure  4.  The  elements  illustrated  this  figure  reflect  the  types  of 
actions  more  typically  associated  with  ISD.  These  actions  can  only  occur  when 
a  detailed  ET  design  exists  defining  the  curriculum,  the  softwa^^e  and  cotaputer 
resources  available,  defining  the  interface  with  the  operational  system,  and 
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Figure  3  Phases  of  Embedded  Training  Analysis 

defining  the  documentation  requirements.  A  critical  concern  is  the  way  the  ET 
courseware  will  be  activated  and  deactivated  and  the  relationship  between  the  ET 
and  the  system. 
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Non-military  use  of  embedded  training  appears  to  be  increasing.  Five  of  the 
most  noteworthy  eommerctal  applications  are:  the  Rochester  Gas  and  Electric 
Company,  the  New  Hampshire  Yanlcee,  the  Arizona  Nuclear  Power  Project,  the 
Ford  Motor  Company,  and  Martin  Marietta  Electronics  Systems  Company  (ESC), 

The  Rochester  Gas  and  Electric  Company  has  established  plant-specific 
training  as  a  portion  of  the  licensed  operator  training  program  for  the  Robert  E. 
Ginna  Nudear  Station.  The  simulator  Is  functionally  Identical  to  the  operational 
system  but  is  maintained  separately  due  to  the  unique  safety  concerns  of  the 
nuclear  Industry.  The  primary  training  subsystems  it\ivoive  stimulation  of  the  Safety 
Assessment  System  (SAS)  and  the  Plant  Process  Computer  System  (PRCS).  The 
trainer  simulates  the  SAS  and  the  PPCS  functions  for  all  modes  of  operation 
including  hot  and  cold  r/cartup,  cold  shutdown,  prestait  checks,  preheating,  power 
maneuvering,  and  normal,  abnormal,  and  emergency  operating  conditions. 

Both  the  New  Hampshire  Yankee  which  runs  the  Seabrook  Station  Nuclear 
Power  Station  In  Seabrook,  New  Hampshire,  and  the  Arizona  Nuclear  Power 
Project  in  Wintersberg,  Arizona,  have  developed  and  adopted  site-specific, 
computer-based  simulation  to  train  plant  operators  to  handle  every  conceivable 
situation  they  might  encounter  in  the  real  reactor.  The  simulator  Is  used  as  one 
training  device  supporting  a  systematic  program  of  training  and  licensing 
operators  In  the  safe  and  efficient  operation  of  a  nuclear  power  station  and 
provides  periodic  refresher  training  and  recertlficatlon,  as  determined  by  the 
Nuclear  Regulatory  Commission  (NRC)  licensing  requirements. 

Ford  Motor  Company  hired  Applied  Science  Associates  (ASA)  to  develop  the 
Automated  Assembly  Une  Trainer  (AALT).  The  AALT  Is  designed  to  provide 
diagnostics  and  Instruction  In  using,  maintaining,  troubleshooting,  and  repairing 
the  automated  assembly  line.  The  system  uses  Interactive  Videodisc  (IVD) 
technology  to  deliver  training  and  job/decislon-ald  data.  The  AALT  can  stimulate 
any  system  component  and  can  be  readily  tailored  to  any  part  on  the  assembly 
line. 

Martin  Marietta  ESC  has  developed  and  Implemented  an  Aerospace 
Planning,  Execution,  and  Control  System  (APECS).  APECS  Is  a  large-scale 
computer  Integrated  manufacturing  effort  for  a  highly  sophisticated  electro-optical 
system.  The  system  is  manufacturing  LANTIRN  electro-optical  night  vision 
systems.  APECS  consists  of  four  major  subsystems:  1)  a  material  resource 
planning  and  control  system;  2)  a  manufacturing  engineering  system  capable  of 
generating  and  maintaining  manufacturing  bills  of  material,  routing,  standards,  and 
process  plans;  3)  shop  floor  control  of  work  priorities,  shop  operations  control, 
and  labor  data  collection,  and;  4)  a  system  that  satisfies  strict  military  standards 
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for  performance  and  work  measurement.  The  facility  Is  automated,  paperless, 
integrated,  interactive,  and  computerized,  it  has  been  nlctcnamed  the  'paperless 

•avtCty  .  I  lie  oyo&ciii  uiivtvo  uio  upeiavui  iC  Call  Up  ueutiiuu  sKuiCnSs,  pnoiogfapns, 

even  video,  showing  the  v/ork  to  be  performed.  The  Manufacturing  Integrated 
Production  Planning  System  (MIPPS)  can  create  text,  tools,  sketches,  and  anything 
else  the  planner  requires.  The  system  has  a  training  data  base  that  is  used  for 
this  to  ensure  that  the  real-time  control  of  the  shop  floor  are  the  keys  to  the 
paperless  factory  of  the  future. 

The  use  of  the  operational  equipment  to  assist  in  the  training  of  employees 
appears  to  be  growing  both  in  military  and  commercial  applications.  Embedded 
Training  trains  an  individual  on  the  operational  equipment  and  is  training  effective. 
The  problem  of  the  transfer  of  training  from  the  training  system  to  the  actual 
equipment  is  not  an  !ssue. 

ET  is  not  the  end  all  to  training  problems.  It  Is,  however,  another  media 
method  available  to  properly  train  individuals. 

Based  upon  this,  tlie  use  of  embedded  training  in  nonmilitary  settings  must  be 
analyzed  in  the  media  selection  process  to  ensure  that  the  most  appropriate 
method  is  utilized.  Some  future  commercial  applicatic  s  of  embedded  training  are 
the  water  works  industry,  assembly  line  companies,  vocational  education,  shop 
training,  user  friendly  computer  training-  word  processing,  spreadsheets, 
databases,  etc.,  school  systems,  communication  systems  and  airline  cockpit 
training  to  name  a  few. 
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Metaphor  and  Instruction 


Instruction  is,  at  least  in  part,  concerned  with  assuring  that  students  gain 
knowledge.  The  canonical  case  of  knowledge  acquisition  is  science.  Scientists,  faced 
with  a  confusing  universe,  attempt  to  order  that  confusion  with  propositions  which 
explain  and  sometimes  predict.  Where  do  such  propositions  come  from?  In  recent  years, 
writers  such  as  Bohm  and  Peat  (1987)  have  suggested  that  metaphor  lies  at  the  heart 
of  scientific  insight.  They  write  that,  'The  essentia!  point...is  that  metaphors  can 
have  an  extraordinary  power,  not  only  to  extend  the  thought  processes  of  science,  but  to 
penetrate  into    yet  unknown  domains  of  reality,  which  are  in  some  sense  implicit  in 
the  metaphor"  (p.  41). 

Historical  background 

What  is  a  metaphor?  How  do  metaphors  differ  from  similes,  analogies,  and 
models?  These  are  not  new  questions  but  they  have  not  been  completely  answered  even 
today.  The  study  of  metaphor  dates  back  to  the  ancient  Greeks.  Aristotle  made  the 
well-known  distinction  between  similes  and  metaphors.  According  to  Aristotle  (1958), 
"Metaphor  consists  in  giving  [a]  thing  a  name  that  belongs  to  something  else..."  (p.  368), 
while  a  simile  is  a  comparison  between  two  things  using  the  words  like  or  as. 
Analogies,  according  to  Aristotle,  are  an  explicit  mapping  of  one  relationship  onto 
another.  Models,  a  relatively  new  form  of  expression,  are  concrete  and  valid 
representations  of  the  structure  of  a  device  or  system  (Mayer,  1989).  Metaphors, 
similes,  analogies  and  models  all  share  some  qualities;  they  all  evoke  some  kind  of 
comparison;  they  all  use  one  thing  to  shed  light  on  another;  and  they  all  rely 
ultimately  on  language.  Metaphors  differ  from  similes,  analogies,  and  models  in  some 
ways,  however.  First,  in  using  one  name  for  another,  as  Aristotle  would  put  it,  one 
comes  to  see  one  thing  in  terms  of  another.  Thus,  users  of  graphic  computer  interfaces 
who  put  "files"  in  "folders"  may  come  to  actually  believe  that  the  files  are  in  folders. 
Another  aspect  of  metaphors  is  that  ground  of  the  comparison  may  be  difficult  to  make 
explicit.  The  metaphor  "Chevrolet  is  the  heartbeat  of  America",  although  readily 
understandable,  is  not  readily  explicable.  Additionally,  metaphors  must  relate  things 
that  are  not  overtly  similar.  One  can  say  that  Math  101  at  the  University  of  Kansas  is 
like  (or  analogous  to)  Math  1  at  some  other  university,  but  one  cannot  say  Math  101  is  a 
metaphor  for  Math  1.  The  final  point  to  be  made  about  metaphors  is  that  they  often 
have  a  certain  affective  power  that  is  usually  missing  from  similes,  analogies,  and 
models.  Consider  "Chevrolet  is  like  the  heartbeat  of  America."  Most  people  would 
agree  that  it  is  weaker  than  the  metaphor. 

As  was  mentioned  above  Aristotle  was  one  of  the  first  to  discuss  metaphor.  There 
are  two  points  to  be  made  about  Aristocle's  treatment  of  metaphor.  The  first  is  that 
metaphor  is  seen  as  a  figure  of  speech,  not  a  cognitive  phenomenon.  The  second  is  that, 
for  Aristotle,  metaphor  still  had  an  important  function.  In  the  Rhetoric  (1958) 
Aristotle  asserts  that  "We  learn  above  all  from  metaphor"  (p.  89)  and  that  for  the  poet 
"the  greatest  thing  by  far  is  to  have  a  command  of  metaphor"  (p.  87).  This  point  is 
noteworthy  because  not  all  thinkers  believe  metaphor  is  important.  The  Aristotelian 
position  yielded  to  the  pressure  of  empiricism  and  logical  positivism  (Johnson,  1980), 
and  was  largely  discounted  through  the  mid-1960*s. 

The  alternate  position,  attributable  to  Plato,  is  that  metaphor  is  essentially  a  frill 
or  deception.  This  view  interprets  metaphor  as  a  rhetorical  device  and  therefore  an 
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imprecise  and  often  nnisleading  tool  (Hoffman  &  Honeck,  1980;  Johnson,  1980).  This 
pMDsition  is  typified  in  John  Locke's  (1961)  opposition  to  figurative  language  on  the 
grounds  that: 

if  we  would  speak  of  Things  as  they  are,  we  must  allow  that  all  the  Art  of 
Rhetorick,  besides  Order  and  Clearness;  all  the  artificial  and  figurative 
application  of  Words  Eloquence  hath  invented,  are  for  nothing  else  but  to 
insinuate  wrong  Ideas,  move  the  Passions,  and  thereby  mislead  the  Judgment; 
and  so  indeed  are  perfect  cheat:  And  therefore,  however  laudable  or  allowable 
Oratory  may  render  them  in  Harangues  and  popular  Addresses,  they  are 
certainly,  in  all  Discourses  that  pretend  to  inform  or  instruct,  wholly  to  be 
avoided;  and  where  Truth  and  Knowledge  are  concerned,  cannot  but  be  a  great 
fault,  either  of  the  Language  or  Person  that  makes  use  of  them.  (pp.  507-8) 

Modem  views 

The  modem  study  of  metaphor  can  be  traced  to  the  work  of  I.  A.  Richards  (1936). 
Richards  felt  that  verbal  metaphor  was  a  product  of  a  more  basic  psychological 
mechanism,  rather  than  just  being  a  figure  of  speech.  Richards  contributed  the 
terminology  (e.g.  tenor,  vehicle,  and  ground)  which  remains  in  use  today  in  much 
research,  although  often  in  a  different  sense  from  Richards*  usage.  In  a  metaphor  such 
as,  "Man  is  a  wolf."  ,  man  is  the  tenor  (or  topic  as  most  modem  writers  prefer),  wolf  is 
the  vehicle,  and  the  idea  that  the  men  share  some  qualities  with  wolves  is  the  ground 
of  the  metaphor. 

Max  Black  held  a  similar  view  of  metaphor  to  Richards.  However,  Max  Black 
(1979)  expanded  Richards'  position  and  held  that  every  metaphor  mediates  a 
structural  correspondence  between  a  primary  subject  and  a  secondary  subject  which  are  to 
be  regarded  as  systems  of  belief  rather  lhan  as  things.  The  metaphor  results  in  an 
interaction  between  the  primary  and  secondary  subject  which  aeates  a  new 
understanding  of  both. 

Following  upon  the  work  of  Richards  and  Black,  Lakoff  and  Johnson  (1980) 
demonstrated  convincingly  that  ordinary  language  is  systematically  riddled  with 
metaphor:  "We  use  ontological  metaphors  to  comprehend  events,  actions,  activities, 
and  states''  (p.  30);  and    . .  we  typically  conceptualize  the  nonphysical  in  terms  of  the 
physical . . (p.  59).  The  importance  of  this  book  was  that  it  demolished  the  notion 
that  metaphor  is  some  kind  of  special  language,  limited  to  literary  applications.  It 
became  obvious  that  not  only  is  metaphor  more  than  a  figure  of  speech,  but  it  is  also  a 
ubiquitous  feature  of  human  communication. 

Given  that  metaphors  have  psychological  importance  it  is  not  surprising  that  some 
felt  they  may  also  have  educational  importance.  Ortony  (1975)  cited  three  reasons 
why  metaphors  should  be  helpful  in  education:  they  are  vivid,  they  express  the 
inexpressible,  and  they  are  informationally  compact.  The  vividness  hypothesis  holds 
that  metaphor  presents  information  in  a  particularly  concrete  manner,  thereby  aiding 
retention.  The  inexpressibility  hypothesis  holds  that  some  ideas  which  cannot  be 
expressed  literally  can  be  expressed  metaphorically.  The  compactness  hypothesis 
holds  that  metaphors  "chunk"  infom\ation,  increasing  communicational  efficiency. 

A  related  part  of  Ortony*s  thesis  is  that  metaphor  allows  a  transfer  of  meaning 
from  something  well-known  to  something  less  well-known.  Theorists  such  as  Haynes 
(1975),  Johnson  (1980),  and  Petrie  (1979)  support  this  notion  while  further  suggesting 
that  the  transfer  of  structures  from  one  domain  to  another  nnay  involve  the  creation  of 
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new  cognitive  structures.  Black  (1979)  noted  that,  '\  .  *  every  metaphor  niay  be  said  to 
mediate  an  analogy  or  structural  correspondence"  (p.  31),  a  positio.t  which  supports  the 
notion  that  metaphor  mediates  the  transfer  of  knowledge  from  one  domain  to  another. 
Whether  or  not  this  transfer  of  knowledge  involves  the  creation  of  something  new 
depends  on  the  interactionist  versus  substitutionist  point  of  view  taken. 

Theories  of  Metaphor 

According  to  the  substitution  view,  a  metaphor  is  simply  an  elaborate  way  of 
stating  something  that  could  be  stated  literally.  As  a  result  a  metaphor  is  viewed  as  no 
different  from  a  simile  or  an  analogy  except  Uiat  it  is  typically  more  abbreviated.  To 
assert  that,  "Man  is  a  wolf  is  only  an  alternate,  indirect  way  to  suggest  that  man  is 
vicious  or  predatory.  The  word  "wolf  substitutes  for  the  attributes  vicious  or 
predatory.  Waggoner  (1984)  suggested  that  nearly  all  metaphor  theorists  are 
substitutionists,  whether  they  identify  thenwelves  as  such  or  not. 

The  interaction  view  of  metaphor  suggests  that  metaphor  is  more  than  a  similarity 
which  could  be  expressed  by  literal  language  (Richards,  1936;  Black,  1962,  1978,  1979). 
Interactionists  claim  that  metaphor  actually  creates  a  similarity  that  did  not 
previously  exist  (Black,  1979).  Although  some  psychologists  have  embraced 
interactionist  views  of  metaphor  (e.g.,  Tourangeau  &  Sternberg,  1981;  Verbrugge,  1980), 
Waggoner  (1984)  \>J^s  argued  that  in  operationalizing  their  theories  they  more  closely 
resemble  substitutionists. 

For  educational  purposes  the  substitution  theory,  in  a  strict  sense,  suggests  that 
metaphor  plays  only  a  limited  role  (Green,  1979).  Substitutionists  assert  that  a 
metaphor  only  points  out  a  similarity  which  could  be  expressed  in  other,  more  literal, 
words.  If  true,  it  would  seem  reasonable  to  simply  present  instructional  information  in 
literal  form.  However,  because  the  distinction  between  substitution  theory  and 
interaction  theory  is  not  always  maintained,  even  substitution  theorists  have 
advocated  the  educational  benefits  of  metaphor  (Ortony,  1975).  Despite  the  growing 
respectability  of  metaphor  as  a  cognitive  phenomenon,  the  educational  view  of 
metaphor  has  been  primarily  Platonic:  To  teach  metaphor  as  a  literary  device  has 
been  acceptable  but  to  use  metaphor  as  an  educational  tool  has  been  equated  with 
deception  (Petrie,  1979)  and  has  been  described  as  unnecessary  (Green,  1979;  Miller, 
1976). 

Science  and  metaphor 

The  literal  bias  of  educational  empiricism  has  not  always  prevailed  in  the  field  of 
science.  One  person  who  implicitly  recognized  the  value  of  metaphor  was  Leonardo  Da 
Vinci.  Wintemitz  (1967)  described  Da  Vinci's  obsession  with  metaphor  and  how  it  led 
to  several  inventions.  In  one  instance,  Leonardo  compared  the  spine  with  its  cords  and 
muscles  to  the  mast  of  a  ship  and  recommended,  as  a  pedagogical  device,  the  replacing 
of  muscles  in  models  of  the  spine  with  thin  cords  to  represent  the  lines  of  force.  Thus  Da 
Vinci  demonstrated  his  understanding  of  the  instructional  value  of  the  non-literal 
representation  of  a  domain  of  knowledge. 

During  the  I960's,  investigators  began  to  recognize  the  apparent  importance  of  non- 
literal  imagery  to  the  explanatory  aspects  of  scientific  theory.  For  example,  Hempel 
(1965)  showed  tliat  some  theoretical  terms  cannot  be  reduced  to  literal  observational 
statements.  Thus  the  naive  idea  that  science  was  based  on  the  literal  interpretation  of 
data  was  weakened.  Black  (1962)  and  Hesse  (1966)  refined  the  relationship  between 
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scientific  theories,  models  which  instantiate  those  theories,  and  the  metaphors  which 
express  the  relationship  between  theories  and  models. 

Miller  (1984)  described  the  19th  century  German  scientific  tradition  in  which 
Einstein  was  educated:  "The  matrix  of  science,  philosophy,  and  technology  in  which 
Einstein  was  educated  and  worked  placed  a  high  premium  on  visual  thinking,  a  mode  of 
thought  that  he  preferred  for  aeative  scientific  thinking"  (p.  48).  Einstein's  earlier 
education  included  attending  a  Pestalozzian  school.  Central  to  Pestalozzi's 
educational  philosophy  was  the  concept  of  Anshauung  which  reierred  to  a  customary 
intuition  or  image  (Walsh,  1952).  For  example,  in  the  famous  19th  century  discussions  of 
the  lines  of  electromagnetic  force,    . .  Anshauung  refers  to  the  intuition  through 
pictures  formed  in  the  mind's  eye  from  previous  visualizations  of  physical  processes  in 
the  world  of  perceptions;  Anshauung  is  superior  to  viewing  merely  with  senses" 
(Miller,  1984,  p.  110).  For  Pestalozzi  the  meaning  of  Anshauung  varied  from 
perceptions  of  the  senses  to  the  form  imposed  on  reality  by  the  mind.  Educationally, 
this  meant  that  perceptions  should  always  precede  words  in  education.  Einstein  went 
from  this  Pestalozzian  environment  ii  to  the  19th  centurj-  German  scientific  commur.ity, 
where,  coincidentally,  Anshauung  was  also  a  central  concept.  Though  Anshauung  is 
generally  considered  to  be  untranslatable  but  from  the  previous  quotation  it  is  apparent 
that  metaphoric  image  would  approximate  the  meaning. 

The  importance  of  metaphorical  imaging  cannot  be  underestimated.  One  of 
Einstein's  famous  Gedanken  experiments,  which  led  to  the  theory  of  relativity, 
consisted  of  imagining  himself  traveling  alongside  a  light  wave.  This  is  a  clear 
metaphoric  image,  formed  completely  within  the  tradition  of  Anshauung,  which 
allowed  a  radically  new  interpretation  of  reality.  Nor  was  this  an  isolated  case,  as 
Einstein  used  similar  images  to  arrive  at  other  conclusions,  such  as  the  gravitational 
bending  of  light. 

Empirical  studies 

There  are  two  areas  of  psychological  study  concerned  with  metaphor 
comprehension.  The  first  attempts  to  elucidate  developmental  questions.  The  second 
deals  with  metaphor  processing  by  mature  subjects. 

A  sizable  body  of  research  on  the  developmental  aspects  of  metaphor 
comprehension  and  production  was  reviewed  by  Ortony,  Reynolds,  and  Arter  (1978). 
Its  relevance  to  instruction  should  be  obvious.  If  metaphors  are  not  understood,  or 
understood  with  difficulty,  by  children  then  their  instructional  utility  will  be 
minimal.  Ortony  et  al.  found  that  the  early  developmental  studies  were  plagued  with 
the  usual  difficulties  in  research  on  children.  For  some  studies  like  Billow  (1975)  and 
Winner,  Rosenthiel  and  Gardner  (1976)  the  stimulus  materials  were  criticized  for  being 
obscure  or  atypical.  In  general,  however,  the  studies  showed  that  children  have  some 
ability  to  comprehend  metaphor  although  it  increases  with  age.  Evidence  of  metaphor 
production  was  obtained  for  children  as  young  as  five  years  old  (Billow,  1975)  and  for 
some  tasks,  at  least,  children's  performance  was  as  good  as  that  of  adults  (Centner, 
1977).  In  more  recent  research.  Winner,  McCarthy,  and  Gardner  (1980)  also  found  strong 
evidence  of  use  of  metaphor  in  young  children  and  found  sequential  steps  in  metaphor 
development,  with  production  of  action-based  metaphors  preceding  the  production  of 
perception-based  metaphors.  PoUio  and  Pickens  (1980)  found  a  similar  developmental 
pattern;  recognition  first  followed  by  paraphrase  and  then  explication.  However, 
Waggoner,  Messe  and  Palermo  (1985)  reported  that  children  as  young  as  7  years  of  age 
correctly  interpret  metaphors  when  embedded  in  story  schemata,  suggesting  that  young 
children's  inability  comprehend  metaphors  may  be  experimental  artifacts  due  to 


contextual  problems.  Recent  reviews  of  the  literclare  have  concluded  that,  although 
metaphoric  comprehension  may  be  confounded  by  various  contextual,  linguistic,  and 
developmental  factors  (Vosniadou,  1987),  children  as  early  as  the  pre-school  years 
have  the  competence  to  understand  metaphors  (Winner,  1988). 

A  second  type  of  research  has  focused  on  the  time  needed  to  process 
metaphor  under  various  conditions.  The  goal  of  these  studies  has  been  to 
determine  whether  or  not  metaphors  are  more  difficult  to  process  than  literal 
language.  The  hypothesis  is  that  the  apparently  anomalous  nature  of 
metaphorical  language  should  require  additional  processing  and  therefore 
reaction  times  should  be  slower.  Two-process  theories  derive  from  Kintsch 
(1974).  On  the  other  hand,  if  metaphor  is  not  anomalous,  comprehension  time 
should  be  approximately  the  same  as  literal  language. 

In  general  metaphor  has  not  been  found  to  increase  processing  time.  Ortony 
(1980)  reported  that  processing  time  interacts  with  context:  Where  sufficient 
context  exists,  metaphors  are  comprehended  as  rapidly  as  literals.  Inhoff, 
Lima,  and  Carroll(1984)  supported  this  position  noting  that, . .  even  in 
circumstances  in  which  few  schemata  have  been  activated  by  the  time  the 
target  sentence  is  read,  metaphors  can  be  comprehended  about  as  easily  as 
literals"  (p.  564). 

Harris,  Lahey,  and  Marsalek  (1980)  reported  that  metaphors  can  be 
processed  effectively  even  under  heavy  attentional  disadvantages.  Using  an 
incidental-learning  paradigm,  subjects  were  asked  to  listen  to  sentences 
containing  metaphors,  dead  metaphors,  and  literal  statements.  A  distracting 
task  (counting  the  words  in  each  sentence)  was  embedded  between  sentences. 
On  a  forced-choice  recognition  test,  nonmetaphors  were  recognized 
significantly  less  often  than  metaphors  and  dead  metaphors.  Thus  research 
suggests  that,  for  adults,  metaphorical  language  is  no  more  difficult  to 
comprehend  and  recognize  than  literal  language.  In  fact,  Hoffman  and 
Kemper  (1987)  have  argued  that  the  reaction  time  studies  cannot  resolve  the 
two-process  question,  but  that  most  of  the  evidence  supports  that  idea  that 
there  is  no  special  metaphor  comprehension  process. 

The  memorability  of  metaphorical  language  has  also  been  studied. 
Harris,  Lahey,  and  Marsalek  (1980)  reported  that  metaphors  are  sometimes 
easier  to  remember  than  literal  statements.  Reynolds  and  Schwartz  (1983) 
reported  grea*'       -^orability  for  metaphorically  stated  conclusions  rather 
than  literally      .w.      nes.  Thus,  in  certain  instances  metaphors  appear  to  be 
easier  to  rem^  avl    .an  literal  sentences. 

In  conclusion,  studies  have  indicated  that  young  children  have 
metaphoric  competence,  although  this  may  be  interfered  with  by  a  variety  of 
factors.  In  adults,  there  is  no  evidence  that  metaphors  are  more  difficult  to 
comprehend  than  literal  language  and  there  is  evidence  that,  under  some 
conditions,  metaphors  are  more  memorable  than  literal  statements. 

Instructional  uses  of  metaphor 

Instruction  is  a  special  case  of  communication  where  the  potential  of  metaphor 
appears  formidable.  Norman,  Centner,  and  Stevens  (1976)  noted  that: 
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One  important  component  of  the  process  of  learning  and  teaching  is  that  of 
communication.  The  teacher  has  the  task  of  conveying  a  particular  knowledge 
structure  to  the  student  The  learner  has  the  task  of  deducing  just  what  structure 
is  intended  by  the  teacher,  as  well  as  the  additional  task  of  adding  the  new 
information  to  his  previous  knowledge  in  such  a  way  that  it  can  be  referred  to 
and  used  at  a  later  time.  Many  of  the  problems  of  learning  and  teaching  can  be 
understood  as  problems  in  this  communication  process,  (pp.  186-7) 

Problems  in  the  communication  process  may  be  traced  to  problems  in  the  medium  or 
problems  in  the  encoding  and  decoding  process.  For  example,  Bonar  and  Solo  way  (1985) 
have  shown  that  many  computer  programming  errors  may  be  traced  to  the  inaccurate 
mapping  of  novice  natural  language  knowledge  onto  a  programming  language.  Students 
mistakenly  overextend  the  metaphor  of  language  (as  in  computer  language)  and  make 
numerous  unwarranted  assumptions.  This  indicates  the  tendency  of  students  to  supply 
their  own  metaphors  when  none  are  explicitly  provided  by  the  teacher.  Carrol!  and 
Thomas  (1982)  argued  that  the  application  of  metaphors  to  new  knowledge  domains  by 
students  is  inevitable  and  that  not  all  metaphors  are  equally  efficacious,  therefore 
appropriate  choice  of  metaphors  is  crucial. 

The  casual  use  of  metaphor  by  teachers  is  probably  more  common  than  often 
assumed.  However,  little  research  has  been  reported  to  validate  the  use.  In  one  study, 
Wilson,  Gaff,  Dienst,  Wood  and  Barry  (1975)  found  that  an  important  characteristic  of 
influential  teachers  is  the  frequent  use  of  analogies.  For  the  most  part,  systematic 
attempts  to  use  metaphor  pedagogically  are  rare. 

There  are  at  least  three  way  in  which  metaphor  can  be  used  to  influence  instruction. 
The  first  is  the  teacher's  (or  instructional  designer's)  conception  of  the  educational 
process  and  his/her  role  in  that  process.  The  second  is  the  student's  conceptual  of 
his/her  role.  The  third  is  in  the  mediation  of  the  content  itself. 

Numerous  writers  have  commented  on  the  fact  that  the  metaphors  we  adopt  for 
instruction  can  influence  how  we  function  as  teachers  or  instructional  designers  (Egan, 
1989;  Heitland,  1981;  Kloss,  1987;  Marshall,  1988;  Moore  1987;  Munby,  1986;  Tom,  1984). 
A  particular  case  of  this  can  be  found  in  the  "container  and  conduit"  metaphor  of  mind 
and  language  (Lakoff  and  Johnson,  1980;  Mosenthal,  1987;  Reddy  1979).  If  we  think  of 
language  as  "containing"  meaning  which  it  "transmits"  to  a  "data  storage  device"  like 
long-  or  short-term  memory  our  lessons  will  assume  certain  characteristics.  On  the 
other  hand,  if  we  think  of  brain  faculties  as  muscles  to  be  exercised  and  strengthened 
thereby,  our  instructional  approaches  will  assume  other  forms  Some  metaphors  may  be 
adopted  unthinkingly,  based  upon  the  general  intellectual  assumptions  of  our  time. 
Others  may  be  adopted  deliberately.  In  any  case  it  seem  obvious  that  if  we  think  of 
ourselves  as  "delivering"  instruction,  or  coaching,  or  acting  as  facilitators,  or  serving  as 
travel  agents,  our  instructional  strategies  will  bear  the  stamp  of  our  metaphors. 

The  second  way  in  which  metaphors  can  influence  instruction  is  in  forming  the 
student's  unconscious  or  meta-cognitive  conception  of  his  or  her  role  in  the  instructional 
process.  One  program  which  makes  deliberate  use  of  metaphor  is  Gordon's  Synectics 
(1961,  1971)  which  treats  metaphor  as  a  metacognitive  strategy  and  trains  students  in 
this  strategy.  The  Synectics  curriculum  involves  training  in  personal  analogy,  direct 
analogy,  and  compressed  conflict.  Personal  analogy  connects  mentally  oneself  to  the 
device  or  mechanism  under  study  and  imagining  the  results.  Direct  analogy  is  the 
t>orrowing  of  natural  solutions  for  engineering  problems.  The  idea  of  caissons,  for 
example,  came  directly  from  the  observation  of  shipworms.  Finally,  compressed 
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conflict  derives  from  the  deliberate  juxtaposition  of  incompatibles,  such  as  burning  ice, 
in  order  tc  force  the  thinker  to  go  beyond  the  assunied  domain  of  the  problem.  Sanders 
and  Sanders'  (1984)  course  in  teaching  creativit)'  through  metaphor  is  deriyatiye  from 
Gordon's  Synectics.  The  Synecf':s  curriculum  has  not  been  subject  to  rigorous  evaluation. 

A  more  controlled  set  of  studies  v^^as  performed  by  Paris  and  his  colleagues  (reported 
in  Paris,  Wixson,  and  Palincsar,  1986).  Paris  et  al.  used  metaphors  to  teach  reading 
strategies.  Metaphors  such  as  "reading  detectives"  and  "planning  reading  trips"  v^ere 
used  to  concretely  structure  metacognitive  strategies.  In  addition  to  supporting  reading 
skills,  these  metaphors  stimulated  group  discussions  about  reading  and  thinking. 
Students  exposed  to  this  metaphorical  curriculum  exhibited  significant  improvements 
in  reading  awareness  and  use  of  reading  strategies. 

The  third  v^ay  in  which  metaphors  can  be  used  is  to  mediate  content  itself.  If 
metaphors  express  the  inexpressible,  are  more  vivid  or,  chunk  information  they  should 
have  positive  effects  on  comprehension  and  memory.  Pearson,  Raphael,  TePaske,  and 
Heyser  (1981)  found  that  metaphoric  sentences  were  recalled  better  than  literals  when 
placed  in  unfamiliar  passages,  although  overall  recall  was  unaffected.  Reynolds  and 
Schwartz  (1983),  as  reported  above,  obtained  contrary  results.  They  found  that 
metaphors  placed  at  the  end  of  prose  passages  were  recalled  better  than  literal 
counterparts  and  that  overall  recall  was  also  improved.  Reynolds  and  Schwartz's 
metaphors  served  as  summarizers  which  may  explain  their  better  overall  results. 
Hayes  and  Mateja  (1981)  found  that  metaphors  embedded  in  text  produce  greater 
engagement  than  comparative  organizers.  Subjects  who  read  a  passage  with  embedded 
metaphoric  allusions  produced  more  inferences  and  less  incorrect  information  than 
subjects  given  a  comparative  organizer  first.  Because  this  same  result  was  found  with 
entirely  different  materials  in  an  eariier  study  (Hayes  &  Tiemey,  1980)  they  argued 
that  the  effect  is  reliable.  In  spite  of  these  results,  Becker  and  Carrier  (1985)  and 
Yarbrough  and  Gagne  (1987)  found  that  the  insertion  of  multiple  metaphors  into  a  text 
did  not  increase  learning.  Both  of  these  studies  used  unrelated  metaphors  and  thus  do 
not  speak  the  effect  of  a  single  systematic  metaphor  used  to  structure  a  body  of  content. 

There  is  not  a  body  of  literature  indicating  that  metaphors,  used  haphazardly, 
will  increase  learning.  However,  metaphors  used  systematically  do  have  positive 
effects  on  learning  and  retention  of  content. 

Theoretical  integration 

General  guidelines  for  the  use  of  metaphors  have  been  reported  elsewhere  (Carroll 
and  Thomas,  1982),  but  there  is  also  the  possibility  of  theoretical  integration  of 
metaphors  into  instructional  design  theory,  Reigeluth's  elaboration  theory  (Reigeluth 
and  Stein,  1983)  and  Keller's  motivational  theory  (Keller,  1983)  both  include 
metaphors  (as  a  kind  of  analop^^  into  their  prescriptions.  Both  authors  seem  to  regard 
metaphors  as  adjunct  aids  to  msiruction  rather  than  having  a  central  function. 

An  alternative  view  is  to  see  the  use  of  metaphor  as  a  function  of  the  degree  of 
abstractnesp  of  the  canonical  case  of  the  content  being  taught,  Reigeluth  calls  this 
canonical  case  an  epitome  and  this  serves  as  a  base  for  his  elaboration  theory.  For 
Reigeluth  an  epitome  consists  of  a  small  number  of  ideas  that  are  most  basic  or 
fundamental  to  the  particular  type  of  content.  In  the  case  of  a  conceptually  based 
biology  course  an  example  of  an  epitome  would  be  the  animal  phyla.  For  a  theoretical 
course  such  as  economics,  the  principle  of  supply  and  demand  would  be  an  epitome. 
Reigeluth  gives  no  example  of  an  epitome  for  a  procedural  course  like  statistics.  The 
essential  issue  here  is  whether  a  literal  epitome  is  the  preferred  treatment  under  all 
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conditions.  Since  the  comprehensibility  of  an  example  depends  upon  the  degree  to 
which  its  crucial  characteristics  are  perceptually  salient  or  not,  it  would  seem  that 
there  must  be  cases  in  which  the  abstractness  (or  at  least  the  unobservability)  of  the 
crucial  features  of  the  example  would  render  it  less  tha  useful.  As  a  simple  case 
consider  learning  to  ski.  The  crucial  characteristics  of  body  movement  are  not 
necessarily  external.  A  skilled  skier  must  hold  an  inner  model  of  correct  posture,  etc.  A 
teacher  of  mine  once  described  this  ski  posture  as  holding  a  basketball  in  your  lap 
rather  than  pointing  out  the  correcC  stance  in  literal  terms,  which  an  epitome  would 
require.  Similarly  in  the  Paris  studies  above,  children  were  told  to  think  of  themselves 
as  detectives.  Presenting  good  reading  strategies  in  literal  terms  to  children  would 
require  the  comprehension  of  exceedingly  abstract  ideas.  One  can  easily  imagine  thai 
in  statistics,  econonucs,  and  biology,  the  degree  to  which  a  literal  epitome  would 
facilitate  learning  would  depend  upon  the  degree  to  which  the  students  could 
assimilate  abstraction  from  an  example. 

Thus,  as  a  hypothesis,  one  could  modify  elaboration  theory  in  this  way.  If  the 
canonical  case  of  the  content  is  relatively  concrete  or  salient  to  the  student,  then  use  an 
epitome  to  begin  a  course.  On  the  other  hand,  if  the  canonical  case  is  abstract  or  obscure, 
from  the  student's  point  of  view,  then  use  a  metaphor  to  introduce  the  content. 
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Applying  Linguistic  Analysis  to  Instructional  Design 

The  terminology  of  linguistic  discourse  is  often  applied  to  instructional  design 
situations.  We  speak  of  tutorials  and  dialogues,  interface  languages  and  interactions. 
It  is  rare  to  see  these  terms  taken  literally,  however.  The  discipline  of  linguistics  has 
formal  description  techniques  which  may  allow  us  to  describe  the  structure  of 
computer-based  interaction  with  greater  objectivity.  Such  devices  have  been  applied 
to  stories  (Mandler  and  Johnson,  1977),  the  cinema  (Carroll,  1980),  computer 
interfaces  (Reisner,  1981;  Moran,  1981;  Payne  and  Green,  1986)  and  classroom 
discourse  (Mehan,  1979).  The  application  of  such  formal  devices  to  instructional 
systems  design  is  a  logical  extension  of  such  a  trend. 

This  paper  is  a  short  tutorial  in  formal  grammar  with  speculative  examples  of 
how  it  could  be  used  as  a  research  tool  for  task-analysis,  the  description  of  lesson 
structure,  the  modeling  of  interactive  dialogue,  and  perhaps  the  instructional  design 
process. 

As  an  example  of  the  application  of  linguistic  techniques  to  ISD,  consider 
Gagn^'s  well-knovm  ''events  of  instruction."  It  is  hard  to  evaluate  the  theoretical 
status  of  the  events.  How  could  they  be  empirically  tested?  Are  all  events  necessary 
always?  How  are  they  to  be  related  to  course-level  events,  units,  lessons,  and  sub- 
lessons?  Must  they  be  applied  in  a  fixed  order?  If  not,  to  what  extent  can  the  order  be 
transformed? 

One  way  of  thinking  about  these  problems  is  to  analogize.  The  events  of 
instruction  may  be  compared  to  the  parts  of  speech.  Sentences  are  made  up  of  parts 
just  as  lessons  are.  Sentences  are  infinite  in  variety  but  they  are  constrained 
structurally.  In  other  words,  certain  strings  of  words  are  considered  to  be  well- 
formed  while  others  are  not.  Tlie  description  of  these  constraints  is  a  goal  of 
linguistics.  Instructional  designs  may  be  thought  of  as  sentences  made  up  from 
events.  Some  combinations  of  events  may  be  thought  of  as  well-formed  strings 
while  others  may  violate  our  intuitions  of  well-formedness.  A  grammar  of 
instructional  design  would  assign  descriptions  to  lesson  materials.  A  basis  for 
assigning  descriptions  to  the  structure  of  instruction  may  be  thought  of  as  a 
theoretical  goal  of  instructional  design  research. 

Formal  Grammar 

At  the  mention  of  the  word  "grammar,**  some  people  remember  tedious 
sentence  diagramming  exercises  performed  under  the  scrutiny  of  a  schoolmarm. 
Others  recall  laboring  to  memorize  Latin  verb  conjugations  and  noun  declensions. 
Those  who  write  professionally  think  of  the  handbooks  they  consult  to  ensure  the 
propriety  of  their  usage.  To  the  linguist  however,  grammar  is  not  an  artifact  of 
language  study  nor  an  arbitrator  of  style.  Rather,  the  linguist  views  grammars  as 
tools  to  describe  the  complex  cognitive  system  which  language  is. 

The  use  of  the  plural  "grammars"  is  deliberate:  to  the  linguist  grammar  is  not 
the  monolith  that  it  is  to  man  in  the  street.  Linguists  ardently  debate  the  relative 
merits  of  a  number  of  grammars,  such  as  Case  grammar,  Tagmemic  grammar, 
Transformational-Generative  grammar  and  Traditional  (Latinate)  grammar,  to 
name  a  few.  Power  and  parsimony  are  the  chief  criteria  considered  in  attempts  to 
empirically  determine  which  system  is  superior. 

The  concern  with  parsimony  leads  to  the  formal,  that  is,  symbolic,  property  of 
modern  grammars.  The  ideal  of  power  requires  that  a  finite  set  of  rules  account  for 
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the  infinite  number  of  possible  sentences  which  any  language  has.  One  such 
grammar  is  Phrase  Structure  (PS)  grammar,  which  is  an  algebraic  system  using 
symbols  to  represent  variables.  It  has  rules  which  specify  whether  a  given  string  of 
words  belongs  to  the  set  of  valid  English  sentences.  Moreover,  it  has  rules  which 
break  down  ("rewrite"  in  the  linguistic  terminology)  sentences  into  their 
constituents.  Figure  1  is  a  simple  PS  grammar  of  English. 

Story  Grammar 

One  of  the  first  and  best-known  applications  of  formal  grammar  outside  of 
pure  sentential  syntax  was  Mandler  and  Johnson's  (1977)  paper  on  story  grammar. 
Essentially,  Mandler  and  Johnson  asserted  that  traditional  stories  (fairy  tales,  legends, 
etc.)  have  a  structure  which  can  be  represented  formally.  The  importance  of  this 
claim  rests  on  the  assumption,  borrowed  from  linguistics,  that  such  structures  are 
''psychologically  real."  That  is,  such  grammars  are  represeutations  of  knowledge  that 
experienced  story-readers  hold  in  their  heads.  Mandler  and  Johnson's  grammar 
(Figure  2)  consists  of  a  finite  set  of  primitives  and  a  finite  set  of  recursive  rewrite 
rules  which  are  capable  of  describing  an  indefinitely  large  number  of  stories. 


S  -  NP  +  VP 

NP  -  DET.  +  N  +  (S) 

DET.     {the,  a,  this,  that,  etc.} 

N    dog,  window,  neighborhood,  hope,  etc. 

r  Vi  +  (PP)  1 
Vp->    \  Vt  +  (NP)  \ 
[  be  +  Adj  J 

Vi      walk,  coughed,  spoke,  etc. 

Vt     hit,  cover,  inform,  etc. 

be    am,  are,  is,  was  were,  been,  being 

PP    Prep  +  NP 

Prep    to,  for,  at,  etc. 

Adj    blue,  tall,  old,  etc. 

Figure  1.  A  simple  phrase  structure  grammar  of  English 

There  is  a  need  for  a  short  explanation  of  notational  conventions. 
Parentheses  indicate  optional  items.  Parentheses  marked  with  a  supersaipted  index 
n  indicate  that  the  item  may  occur  one  or  more  times.  Brackets  indicate  mutually 
exclusive  items.  EVENT*  indicates  that  one  terminal  event  is  conjoined  with  one  or 
more  other  terminal  events  within  a  single  higher-level  node.  STATE*  has  a 
similar  meaning.  Three  types  of  connection  are  allowed:  AND,  THEN,  and  CAUSE. 
AND  and  THEN  define  temporal  relations  and  CAUSE  defines  an  explanatory 
relationship.  These  connections  are  rendered  in  lower  case  in  the  figure  to  improve 
readability. 
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FABLE  -  STOFA  and  MORAL 

STORY     SETTING  and  EVENT-STRUCTURE 

f  QTATE*  (!*nd  EVENT*^  1 
SETTING  -  I  EVENT*^  '  | 

STATE*  -  STATE  ((  and  STATE)!^) 

r  and  1 

EVENT*  -  EVENT  ((  ]  then    i  EVENT)i^)  ((  and  STATE)i^) 

I  cause  J 

EVENT-STRUCTURE  -  EPISODE  (( then  EPISODE)!^) 
EPISODE  -  BEGINNING  cause  DEVELOPMENT  cause  ENDING 

f  EVENT*  1 
BEGINNING  -  |  EPISODE  } 

^T>x,T^xT.r     J  SIMPLE-REACTION  cause  ACTION  1 
DEVELOPMENT  -  |  COMPLEX-REACTION  cause  GO AL-PATHJ 

SIMPLE-REACTION  -  INTERNAL-EVENT  ((  cause  INTERNAL-EVENT)") 
ACTION  -  EVENT 

COMPLEX-REACTION  -  SIMPLE-REACTION  cause  GOAL 

GOAL  -  INTERNAL-STATE 

j  ATTEMPT  cause  OUTCOME 
GOAL-PATH  -  I  GOAL-PATH  (cause  GOAL-PATH)i^. 
ATTEMPT  -  EVENT* 


f  EVENT*  1 
OUTCOME  -  I  EPISODE  \ 

r  EVENT*  (and  EMPHASIS) 
ENDING  -  EMPHASIS 
I  EPISODE 

EMPHASIS  -  STATE 


Figure  2.  Story  Grammar 

This  story  grammar  can  be  used  to  define  the  structure  of  a  traditional  story 
(taken  from  Mandler  and  Johnson),  such  as  that  listed  in  figure  3. 
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1.  It  happened  that  a  dog  had  got  a  piece  of  meat 

2.  and  was  carrying  it  home  in  his  mouth. 

3.  Now  on  his  way  home  he  had  to  cross  a  plank  lying  across  a  stream. 

4.  As  he  aossed  he  looked  down 

5.  and  sa\/  his  own  shadow  reflected  in  the  water  beneath. 

6.  Thinking  it  was  another  dog  with  another  piece  of  meat, 

7.  he  made  up  his  mind  to  have  that  also. 

8.  So  he  made  a  snap  at  the  shadow, 

9.  but  as  he  opened  his  mouth  the  piece  of  meat  fell  out, 

10.  dropped  into  the  water, 

11.  and  was  never  seen  again. 


Figure  3.  Traditional  Story 

Figure  4  is  a  tree-structure  (based  upon  Mandler  and  Johnson)  which 
represents  the  traditional  story  with  each  phr2se  assigned  a  structural  description. 
Each  numbered  node  in  the  tree  structure  '"^presents  the  corresponding  phrase  in  the 
story.  The  circles  with  Cs,  A's,  or  T's  in  them  represent  CAUSE,  AND,  or  THEN 
connections.  As  can  be  seen,  this  grammar  is  capable  of  defining  the  structure  and 
therefore  is  empirically  justified.  !f  the  grammar  had  not  been  capable  of  defining 
the  story,  it  would  have  had  to  have  been  modified  as  a  result  of  this  empirical  test. 


Cinema  Grammar 

Carroll  (1980)  reports  an  attempt  to  write  a  grammar  of  the  cinema.  He  builds 
his  attempt  upon  the  assumption,  not  that  cinema  is  a  language  in  the  same  sense  as 
English,  but  rather  on  the  methodological  assumption  that  cinema  has  structure  and 
that  that  structure  may  be  elucidated  using  the  formal  techniques  of  linguisfics. 
Figure  5  represents  a  portion  of  that  grammar.  The  grammar  as  represented  is  not 
complete  and  is  only  presented  to  give  a  flavor  of  the  type  of  analysis  Carroll 
attempted.  As  can  be  seen,  the  grammar  consists  of  a  set  of  recursive  rewrite  rules 
which  define  the  structure  of  scenes  and  events.  The  notation  is  similar  to  the 
previous  grammars,  tne  adequacy  of  this  grammar  is  not  c  cial  to  our  aigument. 
It  suffices  to  point  out  that  grammars  can  be  applied  to  objects  which  have  temporal 
structure  and  relatively  abstract  primitives. 
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SCENE  -  LONGSHOT  +  DETAIL* 
DETAIL  -  DETAIL  +  SCENE  +  DETAIL 
EVENT  -  NOMINAL  +  SEQUENCE 
SEQUENCE  -  ACTION* 

ACTION  -  PREPARATORY-ACTION  +  FOCAL-ACTION 
P-  A* 

F-^  A*  

Figure  5.  A  cinema  grammar 

Computer  Interface  Grammar 

Reisner  (1981)  first  reported  an  attempt  to  apply  formal  grammar  to  a  human 
factors  design  problem — the  design  of  a  command  language  for  an  interactive 
graphics  program.  Reisner  pointed  out  that  human  factors  design  had  tended  to  use 
behaviorist  models  for  its  methodology.  Reisner  argued  that  cognitive  factors  such 
as  simplicity  and  consistency  would  be  relevant  to  the  design  of  ''action  languages" 
such  as  those  used  to  control  computer  functions.  Basically,  Reisner  argued  that 
cognitive  factors  would  influence  the  learnability  of  action  languages  and  that  these 
factors  could  be  best  modeled  by  a  grammar.  She  wrote  a  grammar  consisting  of 
rewrite  rules  similar  in  notation  to  the  examples  from  story  and  cinema  grammar. 
Her  grammar  could  also  be  expressed  in  tree  diagrams  after  the  fashion  of  the 
Backus-Naur  Form  used  in  computer  science.  Taking  Reisner' s  ideas  further,  Moran 
(1981)  gives  an  elaborate  description  of  a  command  language  grammar  and  elucidates 
how  computer-based  activities  can  be  modeled  grammatically  at  four  levels  of 
analysis  from  keyboard  actions  to  intentions.  Payne  and  Green  (1986)  take  this 
approach  one  step  further  b]^  developing  a  task-action  grammar  (TAG)  which  allows 
tasks  to  be  mapped  onto  actions.  The  power  of  Payne  and  Green  methodology  is  that 
cognitive  structures  (tasks)  may  be  directly  related  to  behaviors  (keystrokes)  while 
capturing  such  notions  as  simplicity  and  consistency.  Payne  and  Green's  hypothesis 
is  that  users  can  recognize  action  language  consistencies  across  tasks  and  therefore 
can  cognitively  structure  those  tasks  at  a  more  abstract  (i.e.,  simpler)  level  rendering 
the  learning  process  easier.  Thus  a  TAG  makes  predictions  about  the  relative 
learnability  of  functionally  comparable  interfaces.  These  predictions  are  amenable  to 
empirical  testing.  Such  information  is  useful  to  the  software  designer,  but  it  also 
could  serve  as  a  task  analysis  methodology  for  instructional  designers  who  need  to 
predict  learning  difficulties  with  computer  interfaces. 

Lesson  Grammar 
Mehan  (1979)  reported  a  description  of  the  structure  of  actual  lessons 
presented  by  a  classroom  teacher.  Mehan  asserted  that  the  activities  of  the  classroom 
seemed  not  to  be  under  immediate  stimulus  control.  It  is  well-known  that  actions 
under  immediate  stimulus  control  can  be  described  by  a  finite  state  automaton  (a 
Markov  device),  but  behavior  of  a  certain  degree  of  complexity  requires  at  least  a 
context-free  grammar.  Mehan's  work  is  important  to  the  present  paper  in  that  it 
demonstrates  the  use  of  a  formal  grammar  in  an  area  closely  allied  to  computer- 
based  instruction.  If  lessons  as  complex  as  those  delivered  by  an  actual  teacher  can  be 
effectively  modeled  with  a  grammar  it  would  seem  that  lessons  delivered  under  the 
controlled  conditions  of  CBI  could  also  be  so  described.  Figure  6  presents  a  part  of 
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Mehan's  grammar.  As  can  be  seen,  lessons  consist  baskally  of  three  parts.  The  main 
part,  the  Middle,  consists  of  topically  related  sets.  Thes^  sets  are  interaction 
sequences  which  consist  of  an  initiation,  a  response,  and  an  evaluation.  An 
evaluation  can  be  an  interaction  sequence  itself,  so  the  internal  structure  of  these 
interactions  can  be  arbitrarily  complex.  The  implications  of  this  grammar  for  the 
desaiption  of  CBI  are  obvious.  Can  it  fully  describe  existing  CBI?  This  question  is 
empirical  and  can  only  be  satisfied  by  the  actual  desaiption  of  existing  materials. 


Lesson      Opening-phase  +  Instructional-phase  +  Closing  -phase 
Opening,  closing-phase     Directive  +  Informative/  Informative  +  Directive 
Instructional-phase  -  Topically-related-set  (=TRS)  -f  (Topically-related-set)^ 
TRS     Basic  +  Conditional-sequence  (or  Interactional-sequence) 
Interactional-sequence  -*  Initiation  +  Reply  +  Evaluation 

Evaluation  Interactional-sequence  

Figure  6.  A  lesson  grammar 

Events  of  Instruction 

As  an  example  of  the  application  of  linguistic  techniques  to  ISD  consider 
Gagn^'s  well-known  "events  of  instruction"  as  illustrated  in  figure  7.  It  is  hard  to 
evaluate  the  theoretical  status  of  the  events.  How  could  they  be  empirically  tested? 
Are  all  events  always.necessary?  How  are  they  to  be  related  to  course-level  events, 
units,  lessons,  and  sub-lessons?  Must  they  be  applied  in  a  fixed  order?  If  not,  to  what 
extent  can  the  order  be  transformed? 


1.  Gaining  attention 

2.  Informing  learner  of  lesson  objective 

3.  Stimulating  recall  of  prior  learning 

4.  Presenting  stimuli  with  distinctive  features 

5.  Guiding  learning 

6.  Eliciting  performance 

7.  Providing  informative  feedback 

8.  Assessing  performance 

9.  Enhancing  retention  and  learning  transfer 
Figure  7.  The  events  of  instruction 

One  way  of  thinking  about  these  problems  is  to  analogize.  The  events  of 
instruction  may  be  compared  to  the  parts  of  speech.  Sentences  are  made  up  of  parts 
just  as  lessons  are.  Sentences  are  infinite  in  variety  but  they  are  constrained 
structurally.  The  description  of  these  constraints  is  a  goal  of  linguistics.  Lessons  may 
be  thought  of  as  sentences  made  up  from  primitive  events.  Primitive  events  may  be 
grouped  structurally  under  larger,  more  abstract  events.  Gagn6's  events  are  of  the 
more  abstract  type  since  they  do  not  specify  actual  lesson  activities.  An  example  of 
the  transformation  of  the  events  of  instruction  into  an  instructional  grammar 
applicable  to  CBI  is  given  in  figure  8. 
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Instructional-Unit     Beginning  +  Middle*  +  End 

Beginning  GA(gaining  attention)  +  IL(informing...)  +  SR(stimulating  recall) 
Middle*  -  Middle  +  (Middle)i^ 

Middle  -  PS(presenting.,.)  +  GL(gmding...)  +  EP(eliciting...)  +  PF(..ieedback) 
End    AP(assessing...)  +  EUenhandng...) 

bright  color,  loud  sound,  motion^  etc> 
Figure  8.  A  grammar  of  instruction 


Note  that  this  grammar  makes  empirical  claims  about  the  order  of  the  events 
and  their  optionality  or  lack  thereof.  Since  lessons  often  consist  of  more  than  one 
objective  they  may  be  arbitrarily  complex.  This  grammars  suggest  how  some  of  that 
complexity  may  be  handled  formally.  No  claim  is  made  that  this  grammar  is 
descriptively  adequate.  In  fact,  it  surely  is  not.  Les.wns  may  consist  of  multiple 
instructional  units.  Several  things  may  be  presented  before  guiding  and  eliciting 
take  place.  This  grammar  does  not  capture  these  facts.  However  it  does  indicate  the 
kinds  of  for  nal  descriptions  which  can  be  accomplished.  The  advantage  of  such 
formalisms  is  that  actual  instructional  designs  may  be  checked  against  this  grammar 
and  validated  for  their  "instructional  grammaticality."  Just  as  native  speakers  can 
recognize  sentences  that  are  not  well-formed,  experienced  learners  presumably  ca  . 
recognize  lessons  that  are  ill-formed.  Empirical  hypotheses  about  such  lessons  could 
be  tested.  Do  learners  learn  less  from  "ungrammatical"  lessons?  Do  learners  slow 
down  when  presented  v^th  lesson  events  which  violate  their  expectations  of 
normality?  These  are  a  few  of  the  questions  which  could  be  investigated  with 
structural  descriptions  of  the  type  provided  by  grammars. 

A  Grammar  of  Instruction  Design 

A  common  statement  heard  in  ISD  circles  is  that  instructional  design  is 
"iterative."  What  this  means  is  that  designers  iterate  through  a  process  of  analysis, 
synthesis,  and  evaluation  when  producing  instructional  materials.  That  this  model 
is  not  perfectly  satisfactory  is  not  news.  Gagn^  and  Briggs  (1979)  state  that  'This 
cyclical  or  iterative  nature  of  the  process  is  real,  but  its  details  cannot  be  shown 
accurately  in  advance,  either  by  feed-forward  or  feedback  loops  in  diagrams  or  by 
arrows  connecting  numbered  stages  in  a  list."  (p.  20-21)  One  would  expect  that  if  the 
process  truly  is  iterative,  empirical  evidence  for  that  "reality"  could  be  collected.  On 
the  other  hand  if  design  is  not  iterative,  perhaps  the  cyclical  metaphor  is  just  the 
most  powerful  model  that  can  be  applied  given  the  available  descriptive  language. 

Alternative  metaphors  are  availabb^  though.  There  are  those  who  have 
noted  that  design  seems  to  be  like  a  "conversation"  with  a  problem  situation  (Schon, 
1983).  A  conversation  model  of  instructional  design  is  not  iterative.  Just  as 
conversations  seem  to  develop  opportunistically  based  upon  speech  acts  and 
responses,  designers  in  action  seem  to  act  alternatively  on  higher-level  and  lower- 
level  problems,  working  on  what  seem  to  afford  most  chance  of  success  at  any  given 
moment  (Stauffer  and  Ullman,  1988).  Linguists  have  attempted  to  characterize  the 
structure  of  conversations  through  discourse  analysis  (e.g.,  Stubbs,  1981}.  The  means 
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by  which  discourse  structures  are  represented  is  by  grammars,  much  like  in  Mandler 
and  Johnson  and  Mehan  above.  If  the  design  process  is  indeed  like  a  conversation 
then,  contrary  to  Gagn6  and  Briggs,  its  detail  can  be  shown  in  advance,  although  its 
actual  trajectory  cannot  be  predicted.  As  an  example  of  this  possibility,  the  following 
design  grammar  is  given.  The  minimal  interactive  imit  of  design  is  hypothesized  to 
be  the  transaction  which  consists  minimally  of  two  events,  an  initiation  and  a 
response.  Other  types  of  events  are  feedback  and  interfaces.  Interfaces  are  events 
which  themselves  initiate  another  transaction.  Events  consist  of  acts  which  are 
actual  behaviors  performed  by  instructional  designers.  Examples  of  acts  are 
collecting  needs  data,  doing  task  analysis,  evaluating  results,  etc.  A  sample  of  such  a 
grammar  is  given  in  Figure  9. 


Design  Session  (Transaction)^ 
Transaction  -  I  +  R  +  (I/F)  -h  (F) 
I/F  Transaction 

I     do  needs  analysis,  do  task  analysis, ... 
R     needs  data,  task  representation, ... 
I/F     evaluate  results, ... 
F  -*  student  behaviors, ... 


Figure  9.  A  grammar  of  instructional  design 


The  first  thing  to  say  about  this  grammar  is  that  it  is  obviously  inadequate. 
Design  projects  consist  of  more  types  of  sessions  and  transactions  than  are 
represented.  Some  of  those  activities  involve  interactions  with  team  members, 
reading  research  reports,  attending  conferences,  removing  constraints,  etc.  However, 
it  does  illustrate  the  kind  of  structural  description  which  goes  beyond  simply  saying 
that  ISD  is  "cyclical."  A  cycle  is  a  model  of  design.  It  may  be  the  best  that  can  be  done 
if  one  restricts  oneself  to  representing  design  with  informal  language.  As  is  well- 
known  though,  science  has  generally  been  advanced  by  using  formal  devices.  There 
is  no  reason  why  the  science  of  the  study  of  instructional  design  processes  should 
handicap  itself  by  restricting  itself  to  informal  language.  In  fact,  a  precedent  for  our 
argument  can  be  found  in  Knowlton's  classic  1966  article  "On  the  definition  of 
'picture.'"  Knowlton  justified  his  attempt  to  develop  a  metalanguage  for  talking 

about  pictures  with  a  quote  from  Antoine  Lavoisier. 
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...we  cannot  improve  the  language  of  any  science  without  at  the  same  time 
improving  the  science  itself;  neither  can  we.  on  the  other  hand,  improve  a 
science  without  improving  the  language  or  nomenclature  which  belongs  to  it. 
(p.  157) 

Summary 

The  ideas  presented  in  this  paper  are  far  from  complete.  Tne  possibilities  of 
formal  grammar  as  an  instructional  design  device  are  unknown.  At  least  it  will 
facilitate  the  identification  of  testable  hypotheses  about  optimal  instructional 
sequences.  Perhaps  it  will  shed  light  on  ISD  from  a  theoretical  standpoint  but  will 
have  no  practical  application.  Perhaps  it  can  be  modified  to  help  designers  generate 
lessons  more  fluently.  Perhaps  such  grammar  will  allow  the  computer-generation  of 
indeterminately  large  numbers  of  different,  acceptable  lesson  structures  which  can  be 
fleshed  oi  t  by  subject  matter  experts.  The  automation  of  instructional  design  will 
require  at  ^he  very  least  some  formal  theory  of  instruction.  The  most  powerful 
formal  theories  known  are  grammars. 
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Educators  are  no  longer  content  to  place  all  their  emphasis 
on  the  environment  and  the  materials  being  used  by  students  to 
guarantee  success.  If  a  teacher  wishes  to  present  a  new  concept, 
concern  is  now  being  focused  on  how  best  to  present  that  concept 
to  the  student. 

The  popularity  of  the  personal  computer  in  the  schools  and 
in  training  facilities  has  made  the  possibility  of  personal 
education  a  reality.   It  is  now  possible  to  diagnose  educational 
weaknesses  and  to  prescribe  certain  types  of  remediation  by  way 
of  the  microcomputer;  this,   in  turn,  frees  the  teacher  to  deal 
with  other  students  in  class  whi'^e  the  one  having  difficulty  with 
an  educational  concept  is  leariiing  through  his  or  her  interaction 
with  the  computer. 

The  most  recent  development  of  interactive  video  has  made 
the  computer  even  more  versatile  by  pairing  it  with  visuals 
provided  by  video  cassette  recorders  or  by  slide  systems.  The 
combination  of  visuals  and  computer-generated  lessons  has  made  an 
already  confusing  situation  even  more  so;  educators  must  now  make 
many  more  decisions  about  the  best  way  to  present  their  materials 
to  learners.  One  of  these  teacher  decisions  deals  with  the  amount 
of  and  with  the  kind  of  elaboration  needed  to  provide  a  desired 
degree  of  student  achievement  while  maintaining  a  realistic  time 
frame  for  the  instructional  process.  This  study  deals  with  the 
question  of  elaboration  and  with  the  efficiency  of  instruction  in 
an  attempt  to  find  the    best  way  to  present  material  to  learners. 

The  major  weakness  in  the  human  information  processing  system  is 
the  short-term  memory.  There  are  two  reasons  for  this  weakness. 
First,  the  information  is  only  present  in  short-term  memory  for 
about  ten  stconds  (Murdock,   1961)  unless  it  is  rehearsed.  Second, 
the  limited  capacity  of  the  short-term  memory  extends  from  five 
to  nine  uniti.^  of  information  (Miller,   1956,  Simon,   1974).  In 
order  to  tranv  fer  information  from  short-term  memory  to  long-term 
memory  it  must  be  coded.  Coding  is  a  transformation  process  in 
which  new  intimation  is  integrated  in  various  ways  with  known 
information,  having  information  in  long-term  memory  is  of  no  use 
if  it  cannot  '^e  retrieved.  The  ability  to  remember  or  retrieve 
information  from  long-term  memory  often  depends  on  establishing  a 
good  retrieval  cue. 

Recognizing  the  limitations  of  short-term  memory  and  the 
need  for  providing  good  retrieval  cues  is  a  prerequisite  for  good 
instruction.  Any  strategy  or  manipulation  taking  place  during 
instruction  can  alter  the  cognitive  structure  and  strengthen  the 
pathway  to  the  information  to  be  learned  and  remembered.  Mayer 
(1980)  reports  that  researchers  have  recently  referred  to  this  as 
the  learning  strategy  hypothesis:   "Mathemagen i c  activities  aimed 
at  making  the  learner  actively  integrate  new  information  with 
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existing  knowledge  affect  the  encoding,  storage  and  eventu-^l  use 
of  new  material  on  performance  tests"  (p. 770). 

Many  activi^.ies  have  been  developed  to  help  the  learner 
integrate  new  information  with  existing  knowledge,   such  as 
advance  organizers,  questioning  strategies,  feedback,  and  review 
procedures.   Each  of  these  activities  provides  for  some  degree  of 
elaboration.  Elaboration  facilitates  retrieval   in  two  ways. 
First,   it  provides  alternate  retrieval  pathways  along  which 
activation  can  spread  (Andersoi,   1976).  Second,   it  provides  extra 
information  from  which  answers  can  be  constructed  (Reder,  1982). 
Although  much  research  has  been  done  with  the  above  activities, 
questions  still  remain:   (a)  How  much  elaboration  is  needed  to 
obtain  the  desired  information?  (b)  Which  mathomagenic  activities 
provide  the  best  means  of  gaining  information?  and  (c)  How  much 
time  should  be  spent  on  these  activities  to  optimize  information 
acquisition?  This  study  added,   in  a  step-like  fashion,  an 
increasing  degree  of  elaboration  t.o  an  instructional  unit  to 
determine  whal:  effect  it  would  have  on  a  posttest  performance 
which  would  measure  different  levels  of  educational  objectives. 
The  activities  which  provided  the  elaboration  were  embedded 
questions,  computer-generated  feedback  and  overt  rehearsal 
strategies . 

The  primary  purpose  of  this  investigation  was  to  assess  the 
effect  of  varying  degrees  of  elaboration  in  a  prose  format  using 
computer-assisted  instruction  with  accompanying  slides  on 
post- test  scores  of  college  age  learners. 

By  exploring  elabor'ation  strategies  and  their  relationship 
with  achievement,   it  was  hypothesized  that  a  significant 
contribution  could  be  made  to  student  achieve^^ent  in  terms  of  how 
people  learn  and  how  instruction  could  be  organized  so  that 
retrieval  of  various  types  of  knowledge  might  be  facilitated. 


This  study  was  designed  to  test  the  following  null 
hypotheses : 

HI:  There  will  be  no  differences  in  student 
achiev^ement  scores  among  the  four 
elaboration  formats  (  control ,  embedded 
questions  (TR1 ) ,  computer  generated  feedback 
(TR2),  and  computer  generated  feedback  with  a 
vi  sua!   rehearsal  strategy  ( TR3  ) )  on  tests 
measuring  different  educational  objectives, 

H2:  There  will  be  no  interaction  between  prior 
knowledge  levels  and  elaboration  strategies 
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among  students  in  the  four  treatment 
groups . 

H3:  There  will  be  no  differences  in  lessor 

completion  time  between  types  of  elaboration 
strategies  among  students  on  tests  measuring 
different  educational  objectives. 

H4:  There  will  be  no  difference  in  efficiency 
scores  on  tests  measuring  different 
educational  objectives  by  treatment  group. 

Elaboration 

Lindsay  and  Norman  (1972)  have  argued  for  the  need  to  retain 
information  longer  in  short-term  memory  to  enable  it  to  be 
processed  further  before  entering  long-term  memory.  Retrieving 
and  interpreting  correctly  the  information  from  long-term  memory 
can  be  as  much  a  problem  as  getting  the  information  into 
long-term  memory  originally.  Cues  given  upon  retrieval  direct  a 
person  to  the  information  or  aseist  him/her  in  generating  new 
cues  or  imposing  additional  structure  on  the  information.  The 
more   interconnections  between  information,   ^.he  more  likely  it  is 
that  a  retrieval  cue  will  be  effective;  memory   is  strengthened  in 
proposition  to  the  redundant  means  utilized  to  retrieve 
information.  Elaboration  is  the  term  used  for  the  process  by 
which  additional  paths  or  links  to  information  are  made. 
Elaboration  uses  connections  to  prior  knowledge,   imaging  and 
inferences,  and  features  from  the  current  context  to  improve 
memory  by  increasing  the  interconnections  and  imposing 
organization  on  the  information  (Anderson,   1980,   pp.  217-218). 
Therefore,  the  more  extra  processing  one  does  that  resuVcs  in 
additional,   related,  or  redundant  propositions,  the  better  one 
will  remember  the  processed   information.  Elaboration  provides 
redundancy  in  the  memory  structure.  Redundancy  can  be  viewed  as  a 
safeguard  against  forgetting  and  an  aid  to  fast  retrieval. 

The  learning  strategy  hypothesis  states  that  mathemagenic 
activities  aimed  at  maKing  the  learner  actively  integrate  new 
information  with  existing  knowledge  affect  the  encoding,  storage, 
and  eventual  use  of  new  material  on  pe^forma.ice  tests  (Mayer, 
1980).   Elaboration  techniques  play  an  important  role  ir.  the 
learning  strategy  hypothesis.   In  general,  past  work  in  this  area 
has  followed  one  of  two  paradigms;  associative  slaooration 
techniques  using  artificial  verbal  materials  or  integrative  and 
comparative  elabcation  techniques  using  real  world  materials. 
The  assocTati\'e  elaboratior>  procedure  involves  making  a  visual 
and/or  verbal  association  between  the  two  stimuli.  Comparative 
elaboratior^  occurs  wher^  a  learner  actively  explains  the  relation 
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between  two  concepts  in  the  text;   integrative  elaboration  occurs 
when  the  learner  explains  the  relation  between  a  concept  in  the 
text  and  some  concepts  already  in  the  learner's  memory.  These 
techniques  differ  from  associative  elaboration  mainly  in  degree; 
they  require  more  than  a  single  mnemonic  sentence  or  image. 

In  a  study  by  Mayer  (1980),  the  elaboration  techniques  were 
used  to  enhance  learning  of  new  technical  information.  His 
results  provided  consistent  evidence  that  elaboration  techniques 
can  influence  the  outcome  of  learning,  even  when  the  material  to 
be  learned  is  in  instructional  text.  His  results  were  strongest 
for   low-ahility  subjects  suggesting  the  need  to  ensure 
integration  encoding  processes  via  instructional  manipulation  is 
strongest  for  learners  who  would  not  normally  use  these 
strategies  for  meaningful   learning.  Thus  there  is  positive 
support  for  the  learning  strategy  hypothesis,  which  states  that 
how  the  learner  encodes  the  material   influences  what  is  learned. 

Because  appropriate  prior  knowledge  and  practice  in  using 
that  knowledge  to  make  meaningful  relationships  are  essential  for 
elaborative  activities  the  use  of  elaboration  may  not  be 
effective  in  improving  learning  and  enhancing  retention  and  may 
even  decrease  achievement  (Bransford,  Stein,  &  Vye,  1982). 

The  rehearsal  strategies  used  in  this  study  were  designed  to 
provide  various  degrees  of  elaboration  by  focusing  attention  and 
allowing  time  for  incoming  information  to  remain  in  short-term 
memory  long  enough  to  be  elaborated  upon  and  encoded  for 
long-term  memory. 


In  this  study,  the  subjects  were  148  undergraduate  students 
(69  males,   79  females)  enrolled  at  the  Altoona  Campus  of  Penn 
State  University. 

The  instructional  content  developed  by  Dwyer  (197?) 
consisted  of  a  2,000  vjord  instructional  unit  describing  the  parts 
of  the  hupan  heart,  the  internal  functions  of  those  parts,  and 
th(='  interrelated  simultaneous  functioning  of  those  parts  with  37 
visuals  of  the  heart.  This  was  later  revised  by  streamlining  the 
presentation  and  reducing  the  amount  of  incidental  content.  The 
revised  script  contained  approximately  1800  words  and  utilized  19 
visuals  of  the  heart.  This  study  used  the  approxirratel  y  1800  word 
revised  script,   but  employed  33  of  the  original  37  2x2  slides 
from  the  original  2000  word  Dwyer  instructional  unit.  The  visuals 
employed  consisted  of  simple,  colored  line  drawings  depicting  the 
form  and  relative  locations  of  the  parts  of  the  heart  as  they  are 
mentioned  in  the  instruction.  The  illustrations  featured  printed 
words  and  arrows  focusing  student  attention  on  the  critical  parts 
being  described.  The  visuals  were  comprised  of  blue  lines  on  a 
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pink  background,  and  color  was  used  solely  to  provide  color 
contrast.  An  example  of  the  visuals  used  in  the  instructional 
unit  is  shown  in  Figure  1. 

The  instructional  treatments  were  presented  via  an 
interactive  video  slide  system  comprised  of  the  following 
equipment:   an  Apple  lie  microcomputer,  a  color  monitor  with  24 
lines  and  40  columns,  and  a  Telex  Caramate  4000  slide  projection 
system.  All  subject  responses  during  instruction  were  restricted 
to  siitiple  keyboard  inputs,  with  the  exception  of  treatment  three 
where  an  overt  visual   rehearsal  method  was  used  in  addition  to 
the  computer  responses. 


The  Verbal  Tests 

Two  of  the  four  posttests  developed  by  Dwyer  (1972)  and 
revised  by  Lamberski   (1980)  were  utilized  in  this  study.  A 
description  of  the  two  tests  as  adapted  from  Dwyer  (1978,  pp. 
45-47)  follows. 

Drawing  Test. 

The  objective  of  the  drawing  test  was  to  evaluate  student 
ability  to  construct  and/or  reproduce  items  in  their  appropriate 
context.  The  drawing  test  (N  =  20  items)  provided  the  students 
with  a  numbered  list  of  terms,  for  example,  superior  vena  cava, 
aorta,  tricuspid  valve,  and  pulmonary  vein,  corresponding  to  the 
parts  of  the  heart  discussed  in  the  instructional  presentation. 
The  students  were  required  to  draw  a  representative  diagram  of 
the  heart  and  place  the  numbers  of  the  listed  parts  in  their 
respective  positions.   For  this  test  the  emphasis  was  on  the 
correct  positioning  of  the  verbal  symbols  with  respect  to  one 
another  and  in  respect  to  their  concrete  referents.  An  average 
reliability  coefficient  of  .83  was  computed  by  Dwyer  (1978,  p. 
47)  using  the  Kuder-Ri chardson  Formula  20  Reliability  coefficient 
on  a  random  sampling  of  previous  studies. 

The  original  drawing  test  was  revised  slightly  in  this  study 
to  test  for  the  effects  of  cued  and  uncued  responses.  Instead  of 
giving  the  learner  the  20  terms  to  identify  on  the  heart  drawing, 
only  10  were  given  and  the  learner  was  asked  to  provide  the 
missing  10  terms  from  memory,  then  locate  them  on  the  heart 
drawing.  The  part  of  the  test  having  the  10  terms  given  to  the 
learner  was  referred  to  as  the  cued  recall  drawing  test  PI  and 
the  part  of  the  test  having  no  given  terms  was  referred  to  as  the 
free  recall  drawing  test  P2. 
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Figure  1.    Sample  of  visuals  used  in  the  instructional  unit. 
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Comp rehens 1  on  Test . 

The  comprehension  test  consisted  of  20  multiple-choice 
items.  Given  the  location  of  certain  parts  of  the  heart  at  a 
particular  moment  of  its  functioning,  the  student  was  asked  to 
determine  the  position  of  other  specified  parts  of  the  heart  at 
the  same  time.  This  test  required  the  students  to  have  a  thorough 
understanding  of  the  heart,   its  parts,   its  internal  functioning, 
and  the  simultaneous  processes  occurring  during  the  systolic  and 
diastolic  phases.  The  comprehension  test  was  designed  to  measure 
the  ability  of  the  individual  to  use  the  information  being 
received  to  explain  some  other  phenomenon.  An  average  reliability 
of  .77  was  computed  by  Dwyer  (1978,  p.  47)  using  the 
Kuder-Richardson  Formula  20  Reliability  coefficient  on  a  random 
sampling  of  previous  studies. 

All  subjects  of  this  study  were  given  the  above  three 
posttests  at  the  completion  of  the  lesson. 

The  Instructional  Treatments 

Each  of  the  four  instructional  treatments  in  this  study 
contained  the  same  instructional  script,  visuals,  terminology 
labels,  and  arrows.  The  treatments  differed  in  the  degree  of 
elaboration  employed.  The  four  degrees  of  elaboration  are 
identified  by  the  treatment  nun^ber.  The  control  group  had  no 
elaboration  method;  Figure  2  is  representative  of  the 
presentation  received  by  the  control  group. 

For  each  of  the  other  three  treatments,  elaboration  was 
added  to  th*-:  script  in  increasing  degrees.  Each  of  the 
elaboration  treatments  had  questions  added  to  the  script.  A  pilot 
test  of  the  heart  material  conducted  during  the  Summer  term  1987 
at  the  Altoona  Campus  of  Penn  State  University  provided  12  areas 
within  the  script  where  students  were  experiencing  difficulty  in 
comprehending  the  information  being  presented.  The  content 
presented  in  these  12  areas  of  the  script  were  found  to  have  been 
evaluated  by  a  total  of  18  questions  on  the  comprehension  and 
drawing  t.ests.   Each  of  these  questions  had  at  least  60  percent  of 
the  student  responses  incorrect  on  the  pilot  test.  Focusing  on 
these  12  areas,  questions  possessing  increasing  degrees  of 
elaboration  were  embedded  in  the  appropriate  sections  of  the 
scr i  pt . 

The  first  treatment,  TR1 ,  embedded  questions  at  the  end  of 
each  identified  section  which  required  the  learner  to  respond  by 
typing  his  response  on  the  computer.  All   responses  were  accepted 
by  the  computer,  and  no  feedback  was  given  as  to  the  correctness 
of  the  response.  Pressing  RETURN  after  typing  in  the  response 
caused  the  computer  to  proceed  to  the  next  part  of  the  lesson. 
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RETURNING  FOQM   ^HE   LUNGS,  3L0O0  ENTERS  TmE 

-•EAPT  THPOUGH  FOUR  PULMONARY  VEINS  ANO   COLLECTS   IN  THE 
■^EFT  AURICLE:    "^hESE  VEIN  OPENINGS  LIKE   THE  VENA  CAVAS , 
^AVE  NO  VALVES.    THE  LEFT  AURICLE  THEN  CONTRACTS  WHEN  IT 
IS  -ULL,    SQUEEZING  8L0OD  THROUGH  THE  MITRAL  VALVE  INTO 
THE   LEFT  VENTRICLE. 


PRESS  RETURN  TO  CONTINUE. 


Figure  2.     Sample  screen  from  tne  control  treatment 
having  no  e^'aboration  added. 
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See  Figure  3  for  an  example  of  the  treatment  one  script  with 
added  quest i  ons . 

The  second  treatment,  TR2,  added  the  same  questions  at  the 
end  of  each  identified  section  as  did  treatment  one,  but  four 
response  options  were  provided  (A,B,C,D)  with  one  of  the  choices 
being  the  correct  answer  and  the  other  three  acting  as 
distractors .  The  learner  pressed  the  letter  of  his  choice  and 
then  the  RETURN  key.  The  response  was  evaluated  by  the  computer 
and  feedback  was  provided  as  to  the  correctness  of  the  response 
to  the  learner.  If  the  correct  answer  was  given  the  learner  was 
told  it  was  correct  and  additional  infoimation  on  that  particular 
topic  was  given  to  provide  further  elaboration.   If  an  incorrect 
response  was  given  the  learner  was  told  it  was  incorrect,  why  it 
was  incorrect,  and  what  the  correct  answer  should  have  been.  In 
giving  the  correct  response  the  same  basic  elaboration  was 
provided  as  was  given  to  the  learner  who  identified  the  correct 
answer.  See  Figure  4  for  an  example  of  the  treatment  two  script 
with  multiple-choice  questions  and  Figure  5  for  a  sample  of  the 
elaboration  provided  as  feedback. 

The  third  treatment,  TR3,  had  the  same  question  structure  as 
the  second  treatment,  but  with  still  further  elaboration  provided 
by  an  overt  visual  rehearsal  strategy.  This  rehearsal  strategy 
required  the  students  to  shade  with  colored  pencils  the  specified 
parts  and  functions  of  the  heart.  The  parts  and  functions  to  be 
shaded  corresponded  to,  as  closely  as  possible,  the  words  that 
wore  used  to  answer  the  questions  in  treatments  one  and  two.  The 
shading  took  olace  on  a  separate  sheet  of  paper  with  three 
provided  drawings  of  the  heart  similar  to  those  appearing  on  the 
lesson  slides.  See  Figure  6  for  an  example  of  the  treatment  three 
script  and  Figure  7  for  the  heart  drawings  used  in  the  overt 
visual   rehearsal  strategy. 

Desi  gn 

The  experimental  design  of  this  study  was  a  randomized 
posttest-onl y  control  group  design.  The  variables  manipulated  in 
the  study  were  the  degree  of  elaboration  present  in  the 
treatments.  There  were  three  treatments  possessing  different 
degrees  of  elaboration  and  one  control  treatment  which  utilized 
no  elaboration,   resulting  in  a  total  of  four  levels  of  the 
elaboration  variable.  The  dependent  variables  were  the 
achievement  scores  on  the  Comprehension  posttest,  the  two  parts 
of  the  Drawing  posttest,  and  time-on-instruction. 

The  physiology  36  question  pretest  was  used  to  label  each 
treatment's  subjects  as  high  or  low  prior  knowledge  level.  This 
was  accomplished  by  designating  those  students  whose  score  was 
above  or  equal  to  the  general  median  score  (21)  to  have  high 
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RETURNING  FROM  THE  LUNGS,   THE  BLOOD  ENTERS  THE 
HEART  THROUGH  FOUR  PULMONARY  VEINS  AND  COLLECTS   IN  THE 
LEFT  AURICLE;   THESE  VEIN  OPENINGS  LIKE  THE  VENA  CAVAS , 
HAVE  NO  VALVES.   THE  LEFT  AURICLE  THEN  CONTRACTS  WHEN  IT 
IS  FULL,    SQUEEZING  BLOOD  THKOUGH  THE  MITRAL  VALVE  INTO 
THE  LEFT  VENTRICLE. 


(Ql  AS  THE  BLOOD  RETURNS  TO  THE  HEART  FROM  THE  LUNGS, 
IT  ENTERS  THE  HEART  THROUGH  THE    VEINS. 


TYPE  RESPONSE  THEN  PRESS  RETURN, 


Figure  3.     Sample  screen  from  treatment  one  with 
questions  added,   but  no  feedback. 
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RETURNING  FROM  THE  LUNGS »   THE  BLOOD  ENTERS  THE 
HEART  THROUGH  FOUR  PULMONARY  VEINS  AND  COLLECTS   IN  THE 
LEFT  AURICLE;   THESE  VEIN  OPENINGS  LIKE  THE  VENA  CAVAS, 
HA,VE  NO  VALVES.   THE  LEFT  AURICLE  THEN  CONTRACTS  WHEN  IT 
IS  FULL,   SQUEEZING  BLOOD  THROUGH  THE  MITRAL  VALVE  INTO 
THE  LEFT  VENTRICLE. 


(Q)  AS  THE  BLOOD  RETURNS  TO  THE  HEART  FROM  THE  LUNGS, 
IT  EWTERS  THE  HEART  THROUGH  THE    VEINS. 


TYPE  LETTER  OF  CHOICE  THEN  PRESS  RETURN. 

A)  MITRAL 

Bi  VENA  CAVA 

C )  AURICULAR 

D)  PULMONARY 


Figure  4.     Sample  screen  from  treatment  two  with 
multiple  choice  questions  added  and 
computer  generated  feedback . 
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Question  #7  -  As  the  blood  returns  to  the  heart  from 

the  lungs,    it  enters  the  heart  through  the   

veins. 

A  )  Mitral 
2  ^   Vena  Cava 
C I  Auricular 
D  )  Pulmonary 

The  correct   response   (D)   Pulmonary  would  result  in  the 
following  feedback  on  the  computer  screen: 

correct: 

THE  "NEW"   CLEANSED  AND  OXYGENATED  BLOOD 
RETURNS  TO   THE  HEART  THROUGH  THE  PULMONARY  VEINS. 

Incorrect   response   (A)  Mitral  would  have  the  following 
feedback  on  the  screen: 

incorrect: 

mitral  refers  to  the  valve  opening  between 
the  left  auricle  and  the  left  ventricle. 

remember,  "new"  blood  returns  to  the  heart  through  the 
p'jlmonary  veins  ^fter  being  cleansed  and  oxygenated. 

Incorrect   response   (Bj   Vena  Cava  would  have  the 
following   feedback  on  tne  screen: 

INCORRECT! 

THE  VENA  CAVA5  DO  ALLOW  BLOOD  TO  ENTER  THE 
HEART  FROM  ALL  PARTS  OF  THE  BODY.    BUT  NOT  FROM  THE 
LUNGS. 

REMEMBER,    "N£W"   BLOOD  RETURNS  TO  THE  HEART  THROUGH  THE 
PULMONARY  VEINS  AFTER  BEING  CLEANSED  AND  OXYGENATED. 

Incorrect  response   (C)  Auricular  would  have  the 
following  feedback  on   cne  screen: 

INCORRECT: 

ALTHOUGH  BLOOD  RETURNING  TO  THE  HEART  FROM 
THE  LUNGS  DOES  ENTER  THE   LEFT  AURICLE  IT  ENTERS  THROUGH 
THE  PULMONARY  VEINS. 

REMEMBER.    "NEW"   BLOOD  RETURNS  TO  THE  HEART  THROUGH  THE 
PULMONARY  VEINS  AFTER  BEING  CLEANSED  AND  OXYGENATED. 


figure  5.   Sample  of  the  elaboration  provided  as 
feedback  to   the  responses  given  m 
treatments  two  and  three. 
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RETURNING  FROM  THE  LUNGS,  THE  3LC0D  ENTERS 
HEART  THROUGH  FOUR  PULMONARY  VEINS  AND  COLLECTS 
LEFT  AURICLE;  THESE  VEIN  OPENINGS  LIKE  THE  VENA 
HAVE  NO  VALVES.   THE  LEFT  AURICLE  THEN 


THE 
IN  THE 
CAVAS , 
CONTRACTS  WHEN  IT 


IS  FULL, 
THE  LEFT 


SQUEEZING 
VENTRICLE 


BLOOD  THROUGH  THE  MITRAL  VALVE  INTO 


«  COLOR  THE  PULMONARY  VEINS  BLACK. 
«  COLOR  THE  MITRAL  VALVE  YELLOW. 

(Q)  ^S  THE  BLOOD  RETURNS  TO  THE  HEART  FROM  THE  LUNGS. 
IT  ENTERS  THE  HEART  THROUGH  THE    VEI/NS. 

TYPE  LETTER  OF  CHOICE  THEN  PRESS  KETURN. 

A  I  MITRAL 

B  )  VENA  CAVA 

C)  AURICULAR 

D)  PULMONARY 


Figure  6.     Sample  screen  from  treatment  three  with  the 
overt  visual  rehearsal  strategy  added  to  the 
multiple  choice  questions  with  feedback. 
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"igure  7.  Heart  drawings  used  in  overt  visual  '^ehearsal 
strategy  in  treatment  tnree. 
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prior  knowledge  level  and  those  below  the  mean  to  have  a  low 
prior  knowledge  level. 

Data  Analysis  Procedures 

The  computer  program  SAS  (Statistical  Analysis  System) 
available  from  the  Computation  Center  at  Penn  State  University 
was  used  to  do  the  statistical  analysis  on  the  data  collected  in 
this  study.  Using    SAS-GLM  (General  Linear  Model),  since  each 
cell  had  different  numbers  of  subjects,  a  multivariate  analysis 
of  variance,  MANOVA ,  was  run  to  test  for  an  overall  effect.  A 
series  of  univariate  ANOVAs  was  then  used  for  one-factor  and 
two-factor  analysis  of  variance  as  a  fciiow  up  to  a  significant 
MANOVA. 

To  test  main  effects  when  there  was  no  interaction,  the 
conservative  Tukey  Studentized  Range  (HSD)  Test  was  used  to 
compare  pairs  of  means. The  Kuder-Ri chardson  20  was  used  to 
generate  the  reliability  coefficient  for  the  multiple-choice 
tests . 

RESULTS 

The  Physiology  TfejSt 

All  subjects  completed  the  physiology  pretest  prior  to 
receiving  their  revspective  instructional  unit.  The  test  was  used 
to  verify  that  proper  randomization  techniques  had  been  used  to 
assign  treatments  to  subjects  by  assuring  homogeneity  of  the 
subjects  with  respect  to  prior  knowledge  related  to  the 
instructional  unit.  The  test  was  also  used  to  determine  the  prior 
knowledge  level  of  the  subjects. 

Stages  of  Analysis 

The  general  analysis  of  the  collected  data  was  done  in  two 
separate  stages  with  each  stage  examining  a  different  aspect  of 
the  study.   In  the  first  stage  a  MANOVA  was  run  to  check  for 
overall  group  and  prior  knowledge  effects  and  to  check  for 
possible  group  by  prior  knowledge  interactions  on  the  criterion 
tests  for  the  four  treatment  groups.  If  significance  was  found  at 
the  .05  alpha  level,  then  follow-up  tests  using  univariate  ANOVAs 
and  Tukey's  HSD  test  were  run.  If  no  significant  difference  was 
found  in  the  first  stage  the  analysis  was  terminated. 

In  the  second  stage,  the  concept  of  efficiency  was 
introduced  with  each  subject's  scores  on  the  criterion  tests 
being  divided  by  his  total  time-on-instruction  to  provide  an 
efficiency  score  for  each  test.  A  MANOVA  was  then  run  to  check 


ERLC 


680 

670 


16 


for  overall  group  and  prior  knowledge  effects  and  to  look  for 
possible  group  by  prior  knowledge  interactions  using  these 
efficiency  scores.  Significant  results  were  then  analyzed  further 
using  follow-up  ANOVAs  and  Tukey  HSD  tests. 

Fi  ndinqs 

Time-on- Instruction 

Finding  that  treatment  TR3  had  significantly  more 
time-on~inscruction  was  no  surprise  considering  the  extra  time 
spent  in  aoing  the  visual   rehearsal  strategy  in  treatment  TR3. 
Treatments  TR1  and  TR2,  although  not  taking  nearly  the  time  of 
TR3,  still  spent  significantly  more  time-on- i nstruction  than  the 
Control  group.  A  significant  difference  in  time  spent  on 
instruction  was  expected  with  the  increasing  degree  of 
elaboration  taking  more  time  to  complete.   In  all  cases  this 
additional  time  spent  on  instruction  resulted  in  higher 
achievement  scores  on  all  the  criterion  tests.  The  decision  to 
adopt  a  treatment  taking  more  time  to  complete  would  depend  on 
the  degree  of  mastery  required  and  the  time  available  for 
1 nstruction . 


The  Comprehension  Test 

Although  the  first  stage  analysis  found  a  :>ignificant 
difference  in  achievement  among  the  four  treatment  groups  there 
was  no  significant  difference  in  achievement  on  the  comprehension 
test.  The  increasing  degrees  of  elaboration  and  the  different 
types  of  elaboration  did  not  increase  the  achievement  on  the 
comprehension  test  significantly. 

The  introduction  of  prior  knowledge  levels   In  the  analysis 
did  find,  however,  that  high  prior  knowledge  subjects  scored 
significantly  higher  than  low  prior  knowledge  subjects  on  the 
comprehension  test.   It  was  felt  that  increasing  the  amount  of 
elaboration  would  help  decrease  the  significant  difference 
between  achievement  scores  of  the  two  prior  knowledge  levels,  but 
this  did  not  occur.  A  possible  explanation  for  this  failure  to 
overcome  these  existing  differences  might  be  the  difficulty  of 
the  test  material.  The  mean  score  on  the  comprehension  test  was 
11.392  out  of  a  possible  20  with  the  high  prior  knowledge 
subjects  having  a  mean  score  of  12.432  and  the  low  prior 
knowledge  subjects  having  a  mean  score  of  10.351.  With  so  much 
room  for  improvement  it  was  unlikely  the  high  prior  knowledge 
subjects  experienced  a  ceiling  effect  ''hich  would  have  allowed 
the  lower  prior  knowledge  subjects  to  close  the  gap  between  their 
scores . 
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The  mean  scores  of  the  high  prior  knowledge  subjects  on  the 
comprehension  test  ranged  from  a  low  of  12.083  in  treatment  group 
TR3  to  a  high  of  12.824  in  treatment  group  TR2,  a  difference  of 
0.741.  The  range  for  the  low  prior  knowledge  subjects  was  from 
8.800  in  the  Control  group  to  a  high  of  11.313  in  treatment  group 
TR3,  a  difference  of  2.513.  Although  the  difference  in  treatment 
group  scores  for  the  high  and  low  prior  knowledge  subjects  were 
not  statistically  significant,  they  did  suggest  that  the 
increasing  degrees  of  elaboration  helped  the  low  prior  knowledge 
subjects  more  than  the  high  prior  knowledge  subjects  on  the 
comprehension  test . 

Although  not  a  significant  difference,  the  high  p^ior 
knowledge  subjects  spent  on  average  36.837  minutes  on  instruction 
while  the  low  prior  knowledge  subjects  spent  on  average  33.948 
minutes  on  instruction.   It  would  seem  that  the  high  prior 
knowledge  subjects  spent  at  least  as  much  time  on  instruction  as 
the  low  prior  knowledge  subjects,   but  they  were  unable  to  improve 
their  achievement  significantly  on  the  comprehension  test 
regardless  of  the  amount  or  kind  of  elaboration  that  was  used. 

A  furth^^  consideration  might  be  that  the  six  questions  that 
vvere  elaborated  upon  in  the  comprehension  test  were  concerned 
with  functions  of  certain  parts  of  the  heart  at  specific  times, 
but  the  increased  elaboration  in  treatment  TR3  was  visually 
oriented  and  more  likely  to  effect  achievement  on  the  drawing 
tests  than  on  the  comprehension  test. 

The  consideration  of  efficiency  scores  in  the  second  stage 
of  the  analysis  also  showed  the  achievement  of  the  high  prior 
knowledge  subjects  to  be  significantly  better  than  the  low  prior 
knowledge  subjects  on  the  comprehension  test.  This  result  pointed 
out  that  any  difference  in  time  spent  on  instruction  between  the 
two  prior  knowledge  levels  was  not  a  significant  factor  in  their 
achievement  scores.  As  ment'oned  earlier,   the  high  prior 
knowledge  subjects  did  spend  more  time  (approximately  three 
minutes)  on  instruction  than  did  the  low  prior  knowledge 
subjects.  Since  efficiency  scores  were  determined  by  dividing 
each  test  score  by  total  t i me-on- i nstructi on  for  each  subject,  a 
significant  difference  in  time-on-instruction  between  two 
subjects  having  comparable  raw  scores  would  result  in  the  subject 
who  spent  more  time-on-instruction  having  a  lower  efficiency 
score.  Since  the  high  prior  knowledge  subjects  still  scored 
significantly  better  than  the  low  prior  knowledge  subjects  when 
efficiency  scores  were  used,  this  would  seem  to  imply  that 
achievement  in^^reased  with  time-on-instruction  so  that  the 
overall   ratio  of  achievement  score  to  time-on-instruction 
remained  about  the  same  for  both  prior  knowledge  groups. 

Using  efficiency  scores  resulted  in  the  first  significant 
difference  between  ^achievement  of  the  treatment  groups  on  the 
comprehension  test.  No  difference  was  found  when  raw  scores  were 
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used,  but  with  the  efficiency  scores  the  treatment  with  the  most 
elaboration  TR3  scored  significantly  lower  than  all  the  other 
treatments.  This  result  was  not  surprising  considering  the  mean 
increase  in  achievement  from  the  Control  group  to  TR3  on  the 
comprehension  test  was  0.98,  but  the  amount  of  time  spent  on 
instruction  more  than  doubled. 

Although  the  difference  in  time-on-i nstruction  between  the 
Control  group  and  treatment  groups  TR2  and  TR3  was  significant, 
their  raw  scores  were  not,  which  resulted  in  the  Control  group 
efficiency  score  being  significantly  higher  than  the  efficiency 
scores  of  treatment  groups  TR2  and  TR3.  The  fact  that  no 
significant  difference  in  time-on-instruction  was  found  between 
treatment  groups  TR1  and  TR2  could  help  explain  why  no  difference 
in  achievement  was  found  between  these  treatments  when  efficiency 
scores  were  used.  The  significance  of  these  results  was  found  in 
the  realization  that  the  considerable  amount  of  time  spent  on 
instruction  in  treatment  TR3  not  only  failed  to  increase 
achievement  proportionately  on  the  comprehension  Lest  but  also 
resulted  in  a  lower  efficiency  score.  This  finding  would  seem  to 
suggest  that  improving  achievement  is  not  simply  a  matter  of 
spending  more  time-on- i nstructi on .  Another  notable  point  was  that 
although  the  difference  in  time  spent  on  instruction  was 
significant  between  Control  and  treatment  TR1 ,  there  was  no 
significant  difference  between  their  efficiency  scores  which 
would  imply  achievement  increased  sufficiently  in  treatment  TR1 
to  offset  the  additional  time  spent.  A  similar  observation  would 
apply  for  treatments  TR1  and  TR2.  This  would  suggest  that  the 
increase  in  achievement  on  the  comprehension  test  gained  by 
increasing  the  degrees  of  elaboration  in  the  first  three 
treatments  was  a  worthwhile  use  of  time  spent  on  instruction. 
Treatment  TR3  did  not  satisfy  this  condition  because  its 
considerable  increase  in  time-on-i nstruction  was  not  offset  by  a 
comparable  increase  in  achievement  on  the  comprehension  test. 

One  final  point  on  the  comprehension  test  results  would  be 
to  consider  the  fact  that  only  6  question  areas  received 
elaboration  out  of  the  20  questions  on  the  test.   It  would  seem 
logical  to  assume  that  any  increase  in  achievement  on  these  6 
questions  would  not  have  the  same  effect  as  increases  in  the  6 
questions  elaborated  upon  in  the  drawing  tests  which  have  only  10 
questions  each. 


Drawing  Test  P1 

The  significant  difference  in  achievement  between  the 
treatment  groups  on  Drawing  test  P1  found  in  the  first  stage  of 
the  analysis  showed  treatment  TR3  to  be  significantly  better  than 
the  other  treatments.  Since  treatment  TR3  had  a  visual  rehearsal 
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strategy  which  was  directly  related  to  the  information  tested  in 
Drawing  test  P1   it  was  not  surprising  that  this  treatment  scored 
significantly  better  than  the  other  treatments  which  did  not  have 
this  additional  visual  component.  Although  the  mean  achievement 
scores  in  the  first  two  treatments  TR1  and  TR2  were  not 
significantly  higher  than  the  score  of  the  Control  group  on 
Drawing  test  P1  they  were  higher  (Control  5.11   to  TR1  and  TR2 
5.49).  This  would  seem  to  possibly  indicate  that  the  elaboration 
in  treatments  TR1  and  TR2,  while  having  some  effect,  did  not 
effect  achievement  like  treatment  TR3  (mean  score  7.55  out  of 
10).  Previous  research  has  shown  that  visuals  such  as  the  line 
drawings  in  this  study  used  in  conjunction  with  prose  instruction 
do  improve  student  learning  and  consequent  ability  to  reproduce 
the  visuals  in  a  drawing  test.  However,   in  a  study  using  similar 
ma\erial  and  treatments  by  Dwyer,   1984    no  significant  difference 
was  found  on  the  drawing  test.  The  fact  that  Drawing  test  P1  was 
a  test  of  cued  recall  evaluating  the  subject's  ability  to 
construct  and/or  reproduce  items  in  their  appropriate  context  was 
probably  the  main  reason  why  treatments  TR1   and  TR2  did  not  show 
much  effect  since  they  did  not  concentrate  on  the  visual  aspects 
of  the  heart  material.  The  increased  elaboration  in  TR2  over  TR1 
was  of  little  consequence  as  the  content  elaborated  upon  was  not 
tested  in  P1 . 

As  with  the  Comprehension  test,  the  high  prior  knowledge 
subjects  did  significantly  better  than  the  low  prior  knowledge 
subjects  on  Drawing  test  P1 .  The  difference  in  mean  achievement 
scores  for  the  high  and  low  prior  knowledge  subjects  were  almost 
identical  for  each  ^?  the  treatments  showing  that  the  elaboration 
had  the  same  effect  on  both  prior  knowledge  levels  on  Drawing 
test  P1 .  Both  prior  knowledge  levels  increased  their  achievement 
as  ohe  amount  of  elaboration  increased  on  the  targeted  material. 

Using  efficiency  scores  resulted  in  the  Control  group 
achieving  significantly  better  than  treatment  TRv3  on  Drawing  test 
PI,  but  no  difference  was  found  between  the  other  treatments. 
This  seemed  to  validate  the  earlier  findings  that  treatments  TR1 
and  TR2  had  very  little  effect  on  achievement  on  Drawing  test  P1. 
The  earlier  mentioned  study  done  by  Dwyer,   1984  used  the  same 
content  and  similar  tests  but  found  no  significant  difference  on 
the  drawing  test.  The  major  difference  between  her  study  and  this 
one  was  the  use  of  computer-generated  feedback  that  was  also 
included  in  TR3.  Pernaps  this  elaboration  while  not  very 
effective  on  its  own  (TR2)  may  interact  with  the  overt  visual 
rehearsal  strategy  to  produce  an  effective  means  of  presenting 
the  mater i al . 
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Drawing  Test  P2 

The  only  difference  between  Drawing  tests  P1  and  P2  was  the 
cues  present  in  P1  were  absent  in  P2 .  The  results  were  similar  in 
that  treatment  TR3  was  again  significantly  better  than  the  other 
treatment  groups  when  raw  scores  were  used  and  considerably  worse 
than  the  other  groups  when  efficiency  sc:res  were  used.  The 
similarity  ends  when  the  treatments  are  compared  for  each  test. 
The  achievement  on  Drawing  tes^  P2  was  the  only  one  where  each 
increase  in  elaboration  resulted  in  a  noticeable  increase  in 
achievement  from  the  previous  treatment.  Achievement  on  the 
Comprehension  test  actually  dropp^  '  in  treatment  TR2  while 
achievement  in  Drawing  test  P1  £      /ed  practically  no  increase 
from  the  Control  to  t*^eatment  TR      Perhaps  the  elaboration  done 
in  treatments  TR1   and  TR2  while  not  improving  achievement  on 
Drawing  test  P1  with  cued  recall  did  provide  the  needed  help  in 
Drawing  test  P2  where  recall  cues  were  not  provided.  The 
rehearsal  strategy  of  typing  responses  and  answering  questions 
present  in  TR1  and  TR2  may  have  helped  to  further  implant  these 
terms  into  long-term  memory.  The  achievement  of  the  prior 
knowledge  levels  on  Drawing  test  P2  were  similar  to  those  on 
Drawing  test  P1  with  the  high  prior  knowledge  subjects  having 
approximately  the  same  difference  in  achievement  o*  ar  the  low 
prior  knowledge  subjects  in  each  treatment.  The  significance  of 
Drawing  test  P2  seems  to  lie  in  the  fact  that  it  did  not  provide 
cues  to  help  the  subject  recall  the  names  for  the  parts  of  the 
heart  so  that  they  could  then  be  located  on  the  drawing  of  the 
heart.  Because  thase  cues  were  not  given  the  subject  had  to 
remember  them  from  the  given  instruction  and  at  this  point  the 
addit-^onal  elaboration  given  in  the  treatments  may  havp*  provided 
some  cues  to  help  the  subject  retrieve  this  information  from 
long-term  memory. 


The  hypotheses  presented  in  this  study  and  the  results  for 
each  are  as  follows: 

H1:  There  will  be  an  insignificant  difference  among  achievement 
scores  of  the  four  presentation  formats,  control,  embedded 
questions  (TR1 ) ,  computer-generated  feedback  (TR2) ,  and 
computer-generated  feedback  with  a  visual  rehearsal  strategy 
(TR3). 

This  hypothesis  was  rejected  as  a  significant  difference  was 
found  between  the  achievement  of  the  four  presentation  formats 
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with  treatment  TR3  having  Lhe  highest  achievement  scores  on  all 
three  criterion  tests, 

H2:  There  will  be  an  insignificant  interaction  between  prior 
knowledge  level  and  type  of  rehearsal  strategy  among  students  in 
the  four  treatment  groups. 

This  hypothesis  was  retained  as  the  subjects  with  high  prior 
knowledge  level  achieved  significantly  higher  scores  on  all  the 
criterion  tests  than  the  low  prior  knowledge  subjects  regardless 
of  the  treatment. 

H3:  There  will  be  i  ns  i  gn  i  f  i  ca,it  differences  in  lesson  completion 
time  between  types  of  rehearsal  strategies  among  students  on 
tests  measuring  different  educational  objectives. 

This  I /pothesis  was  rejected  as  a  significant  difference  was 
found  between  the  time-on-i nstruction  for  each  of  the  treatments 
with  TR3  ?  TR2  >  TR1   >  Control.  As  the  degree  of  elaboration 
increased  the  subjects  spent  more  time-on-i nstruction  even  though 
the  instruction  was  self-paced. 

H4:  There  will  be  an  insignificant  difference  in  efficiency 
scores  on  tests  measuring  different  educational  objectives  by 
treatment  group. 

This  hypothesis  was  also  rejected.  As  the  degree  of 
elaboration  increased  the  time-on-i nstruction  also  increased 
yielding  a  significant  increase  in  achievement  on  the  three 
criterion  tests.  Although  the  test  scores  did  increase 
significantly,  the  efficiency  scores  of  each  treatment  group 
diminished  as  the  amount  of  time  spent  on  instruction  increased 
for  each  of  the  criterion  tests.  Control   >  TR1   >  TR2  >  TR3. 

Conclusions 

The  results  of  this  ^...udy  have  provided  some  interesting 
points  for  consideration  before  planning  any  instruction  using 
increasing  degrees  of  elaboration.  As  with  most  research  these 
findings  can  only  suggest  a  possible  path  to  follow  when 
preparing  instruction,  but  a  path  nonetheless. 

The  fact  that  elaboration  works  seems  to  be  rather  clearly 
shown.  The  effect  of  elaboration  on  high  prior  knowledge  subjects 
can  be  both  helpful  and  harmful  depending  on  the  circumstances  of 
the  instruction.   If  the  content  is  rather  easy  the  time  taken  for 
elaboration  is  probably  not  time  well  spent  for  the  high  prior 
knowledge  subject  and  could  lead  to  boredom  and  a  subsequent 
decrease  in  achievement.  The  effect  of  elaboration  on  the  low 
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prior  Knowledge  subject  for  easier  material   is  much  more  positive 
as  it  allows  this  type  learner  to  acquire  material  that  would  be 
missed  without  the  extra  elaboration.  The  extra  time  spent  on 
instruction  because  of  the  elaboration  also  helps  the  low  prior 
knowledge  learner  to  better  organize  the  material   in  short-term 
memory  which  results  in  better  retention  and  retrieval  from 
long-term  memory.  With  harder  material  the  effect  of  elaboration 
is  going  to  depeind  to  a  large  degree  on  the  amount  of  elaboration 
and  time  spent  on  the  instruction  with  the  lower  prior  knowledge 
learners  requiring  considerably  more  of  both  than  the  higher 
prior  knovsledge  learners. 

The  efficiency  of  instruction  using  elaboration  will 
generally  be  rather  poor  as  more  elaboration  goes  hand  in  hand 
with  more  time  spent  on  instruction.  The  results  of  this  study 
seemed  to  indicate  that  the  gains  in  achievement  were  not 
commensurate  with  the  amount  of  time  spent  on  the  instruction. 

Subjects  with  high  prior  knowledge  will  consistently  score 
better  than  subjects  with  low  prior  knowledge  regardless  of  the 
kind  or  amount  of  elaboration.  On  tests  of  increasing  cognitive 
difficulty  both  groups  will  benefit  about  the  same  from  the  extra 
elaboration . 
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ADULTS'  PERCEPTIONS  OF  CONCEPT  LEARNING  OUTCOMES:  AN 

INIIIMLdlODT  f\tW  Vl9\^»9ifM\yn 


The  purpose  of  this  paper  is  t\A/ofold:  (1)  to  report  on  empirical 
study  of  educators'  perceptions  of  learning  concepts,  and  (2) 
based  on  the  study's  findings  end  a  review  of  cognitive  learning 
literature,  to  arp'je  for  on  expended  view  of  conceptual 
knowledge  and  its  role  in  education  end  training. 


Changing  Views  of  Concept  Learning 

Concepts  are  essentially  categories  that  people  use  to 
organize  their  worlds  and  give  order  end  meaning  to  their 
experiences.  People  learn  concepts  both  by  interacting 
naturally  with  their  environments  and  through  school  and  formal 
learning  experiences  (Brown,  Collins,  &  Duguid,  1989).  Also, 
people  often  spontaneously  invent  concepts  to  solve  local 
problems. 

Over  the  past  few  years,  several  instructional  design  theorists 
have  suggested  revisions  in  the  traditional  view  of  concept 
learning  Gennyson  &  Cocchiarella,  1986;  Wilson,  1986;  Newby  & 
Stepich,  1987;  Tessmer,  Wilson,  &  Driscoll,  1990).  According  to 
these  researchers,  the  well-established  "concept  attainment" 
paradigm  in  instructional  design  literature  (e.g.,  Mari<le,  1976; 
Merrill  &  Tennyson,  1977)  does  not  take  into  account  advances  in 
cognitive  psychology  explaining  how  people  acquire 
conceptual  structures.  Concepts  should  no  longer  be  thought 
of  as  being  completely  defined  by  attributes-based  definitions. 
Many  concepts  may  be  ill-defined  or  "fuzzy",  and  many 
concepts  have  rich  connotations  and  shades  of  meaning 
whose  understanding  is  necessary  to  make  full  use  of  the 
concept.  Recommended  instructional  strategies  include 
greater  use  of  analogies  ,  "prototype*  examples,  and  learning 
strategies,  and  greater  use  of  the  concept  in  a  variety  of 
problem-solving  settings. 


Declarative  and  Procedural  Components  of  Concepts 

Historically,  instructional  design  models  have  addressed  a 
proDlem  common  in  formal  school  settings,  in  which  a  definition 
may  be  learned  by  rote,  and  students  are  never  required  to 
actually  use  the  definition  to  classify  sample  cases.  Largely  as  a 
response  to  schools'  preoccupation  with  inert,  declarative 
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knowledge,  instructional  designers  emphasized  classification 
performance;  that  \s.  given  a  newly  encountered  case,  can  the 
student  identify  that  case  as  being  c  member  of  the  concept 
category?  The  roots  of  viewing  concepts  as  classification  skills 
can  be  traced  back  to  Aristotle,  and  more  recently,  to  Thomdike 
and  Hull's  laboratory  studies  on  human  learning.  Basically, 
students  capable  of  classifying  new  cases  were  sold  to  know 
the  concept.  Gagne  (1985),  following  Anderson's  (1983)  theory 
of  cognitive  leaming,  calls  this  kind  of  performance  "pattern 
matching"  and  treats  it  as  a  kind  of  procedural  skill.  Like  most 
procedural  knowledge,  concept  classification  performance 
can  be  acquired  and  refined  through  repeated  practice  with 
feedback.  Example  classification  strategies  form  a  major  part 
of  traditional  recommendations  for  teaching  concepts,  such  as 
the  use  of  divergent  examples,  proceeding  from  easy  to 
complex  cases,  highlighting  key  attributes,  and  providing 
explanatory  or  elaborative  feedback  to  missed  practice  items. 

Notwithstanding  instructional  design  emphasis  on  procedural 
aspects,  language  experts  have  long  understood  that  in 
addition  to  denotations  (the  set  of  cases  the  term  refers  to), 
terms  or  concepts  also  have  connotations  (the  meaning  or 
properties  suggested  by  the  term).  A  person  who  has  leamed 
what  a  sea  otter  is,  for  example,  should  be  able  to  identify  a 
sea  otter  from  a  harbor  seal,  but  should  also  be  able  to  tell  us 
something  significant  about  tae  animal.  One  could  imagine  a 
student  narrowly  trained  to  classify  coses  but  lacking  in 
declarative  knowledge  about  the  subject.  Such  a  student 
could  "use"  the  pattern-matching  skills  to  Identify  coses,  but  may 
not  hove  the  least  notion  about  what  the  classification  meant 
Equally  disturbing,  the  lock  of  meaningful  understanding  about 
the  category  would  inhibit  the  concept's  effective  use  in  most 
realistic  problem-solving  situations.  The  student  would  probably 
not  understand  when  and  where  to  us©  the  concept 
spontaneously  to  achieve  a  particular  goal,  or  to  draw 
inferences  from  its  meaning.  An  optimal  leaming  of  concepts, 
then,  would  seem  to  include  components  of  both  declarative 
and  procedural  knowledge,  combined  in  a  way  that  mode  the 
concept  meaningful  and  usable  in  real-life  situations. 


Really  Learning  Concepts:  A  Survey  of  Educators 

Persuaded  that  most  educators  would  agree  that  both 
declarative  and  procedural  aspects  ore  important  outcomes, 
we  conducted  an  exploratory  study  to  examine  the  kinds  of 
learning  indicators  educators  would  accept  as  evidence  of 
concept  ieornina.  We  asked  students  (N=56)  in  three  graduate 
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education  classes  to  respond  to  the  following  scenario:  You  ore 
teaching  college  sophomores  of  medium  to  above  average 
intelligence  with  no  prior  knowledge  of  the  topic.  What 
performances  wo  jld  serve  as  indicators  that  students  had  really 
learned  the  concepts  glasnost.  snow  leopard,  and  Justice. 
representing  a  sampling  of  concrete  and  defined  concepts, 
"ubjects  filled  in  blank  spaces  with  their  choice  of  learning 
indicators. 


NUMBER  AND  TYPE  OF  CONCEPT  LEARNING  INDICATORS 

(Total  respondents  =  56;  responses  =  339) 

TOTAL  CONCEPT  LEARNING  INDICATOR  


127  Definitions  and  defining  attributes 

62  Describe  defining  attributes 

38  Give  the  definition 

27  Describe  the  concept 


66  Examples  and  nonexamples 

38  Classify/recognize  cases 

28  Generate  examples 


62  Elaboration 

45  Describe  concept  elements  (not 

defining  attributes) 
5  Teach  the  concept  to  others 

5  Relate  the  concept  to  other  concepts 

4  Distinguish  between  concepts  by 

elements 
3  Explain  the  concept 


84  Use 

22  Describe  effects/implications 

20  Use  in  a  sentence/story 

17  Solve  problems  using  the  concept 

9  Critique  situations  and  actions  using 
the  concept 

9  Write  a  story  about  the  concept 

7  Role-play 
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A  major  finding  is  the  broad  array  of  concept  performances  that 
subjects  listed.  Table  1  summarizes  the  responses  according  to 
type  of  performance  suggested.  The  56  respondents 
generated  339  performances  for  the  3  concepts.  Of  those 
performances,  38  suggested  concept  classification  following 
traditional  instructional  design  models.  However,  a  large 
number  (217)  of  other  performances  like  "discuss",  "compare", 
"describe",  "draw",  "give  an  example"  were  elicited, 
suggesting  the  commonsense  prominence  of  having  a 
declarative  understanding  of  ihe  meaning  of  the  concept.  In 
addition,  using  the  concept  in  a  variety  of  creative  ways,  and 
drawing  inferences  from  it  were  also  thought  to  be  valid 
indicators  of  concept  learning.  Eighty-four  responses 
mentioned  uses  such  as  problem  solving,  predicting  effects, 
role-playing,  and  using  the  concept  in  conversation  or  writing. 

An  inleresting  point  made  by  several  researchers  (e.g., 
Howard,  1988)  and  confirmed  by  our  study's  findings  is  the 
schema-lil<e  role  played  by  many  concepts.  Schemes, 
according  to  models  of  cognitive  processing,  consist  of  a 
whole  network  of  related  knowledge,  centered  around  a 
particular  setting  or  idea.  Many  concepts,  such  as  'justice'  in  our 
study,  are  rich  in  their  associations  with  other  concepts.  The  web 
of  relations  together  may  constitute  a  schema  for  that  concept, 
so  that  concepts  become  schema-like  in  their  complexity. 
While  it  is  important  to  differentiate  between  the  concept  itself 
(the  category  and  its  basic  meaning)  and  its  full-blown  schema, 
an  understanding  of  how  a  concept  is  related  to  other 
knowledge  can  provide  a  basis  for  prescribing  effective  use  of 
the  concept  for  solving  significant  context-based  problems. 
The  instructional  designer's  emphasis  of  "using"  concepts  can 
then  be  expanded  beyond  classification  performance  to 
problem-solving  and  other  divergent  uses,  such  as  making 
inferences  and  developing  analogies.  In  order  to  make 
creative  use  of  concepts,  declarative,  procedural,  and 
mefacognitive  knowledge  must  be  acquired  by  the  learner. 
We  would  recommend  an  instructional  design  model  that 
included  these  components  with  respect  to  concept  learning 
as  well  as  all  looming  outcomes. 


New  Methods  for  Teaching  Concepts 

The  expanded  notion  of  concept  learning  described  above 
has  definite  implications  for  concept  teaching  strategies. 
Summarized  below  ore  several  strategies  that  include  the 
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Using  a  concept  in  different  symbol  systenns  occurs,  for 
example,  when  learners  read  about  the  social  hierarchy  of 
whales  and  draw  sketches  depicting  their  understanding  of 
"hierarchy."  This  process  of  translating  from  one  code  to  another 
--  in  this  case,  verbal  to  visual  -  has  been  termed  transmediafion 
(Suhor,  1982)  and  has  assisted  learners  In  gaining  new  insights 
and  broadened  understandings  about  concepts  (Siegel,  1985). 

Concept  use  instruction  can  proceed  through  several  stages. 
Learners  can  begin  by  paraphrasing  the  concept  definition  of 
attributes  (R.  Anderson,  1973)  and  conversing  about  the 
concepts  (Markle,  1975;  Wilson  &  Tessmer,  1989),  then  move  to 
utilizing  the  concept  in  conversations  and  other 
communi'^cfions,  and  finally  to  using  the  concept  with  other 
declarative  or  procedural  knowledge  to  solve  problems 
(Wilson  &  Tessmer,  1989). 

4  Alternative  strategies  for  dassificatlon  performance.  A 

variety  of  alternate  presentation  strategies  can  be  used  to 
supplement  the  standard  definition-plus- 
examples/nonexamples  strategies  used  in  example 
classitication  training.  Many  oi  these  strategies  have  been 
advocated  by  instructional  theorists  over  the  years. 

For  example,  displaying  coordinate  concepts  in  structural 
outlines  such  as  concept  trees  has  facilitoted  coordinate 
concept  classification  (e.g.,  Tessmer  &  Driscoll,  1986),  and  may 
be  particularly  effective  with  adult  learners  (Bower,  1970).  For 
classification  practice,  concept  games  and  simulations  are  a 
little-used  strategy  that  puts  the  learner  into  scenarios  that  elicit 
classification  performance  in  ways  that  differ  from  standard 
concept  example  practice  (Tessmer,  Jonassen  &  Caverly, 
1989).  Particularly,  concept  simulations  mimic  the  real-world 
situations  in  which  the  learner  will  use  concept  classification, 
increasing  the  probability  of  transfer.  To  create  concept 
examples/nonexamples  that  maximize  generalization  and 
discrimination,  the  Rational  Set  Generator  (Driscoll  &  Tessmer, 
1985;  Tessmer  &  Driscoll,  1986)  can  be  used  to  design  sequences 
of  coordinate  concept  examples  and  nonexamples. 

Hierarchical  displays  facilitate  the  learning  and  organization  of 
the  declarative  component  of  concept  classifying,  while 
rational  sets,  games  and  concepts  facilitate  mastery  of  the 
classification  skills  of  concept  learning  (E.  Gagne,  1985).  When 
used  with  analogies,  and  use  and  inference  practice,  the 
aforesaid  interventions  extend  what  Reigeluth  (1983)  would  call 
the  meaningfulness  of  the  concept. 
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additional  skills  that  encourage  concept  use  and  inference. 

L^Onv^epiS  tOu^i  It  ill  Tnio  UivjUmoi  wvjy  win  mC?  iiiv^iw  iir\»./ly  iv  k^v^ 

used  in  realistic  performance  situations. 

The  obvious  danger  in  broadening  the  view  of  concepts  and 
concept  teaching  is  that  the  notion  will  become  ill-focused  and 
lack  prescriptive  power.  We  believe  that,  on  the  contrary,  a 
broader  approach  to  concept  learning  can  help  bring  our 
instructional  models  back  into  alignment  with  everyday 
teaching  and  learning.  A  continuing  effort  to  articulate 
instructional  strategies  such  as  inferences,  simulations,  scenario- 
based  instruction,  and  case  studies  will  help  concepts  come 
alive  to  learners. 
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This  paper  presents  two  general  approaches  to  the  assessment  of  knowledge 
structures.  The  first  approach  entails  the  building  of  empirical  evidence  to 
support  cognitive  theory.  This  type  of  assessment  is  concerned  with 
attempting  to  prove  the  existence  of  various  knowledge  structures,  that  is, 
evidence  that  leads  to  the  construct  validity  of  these  metaphorical  ideas.  The 
second  approach  is  dependent  upon  the  first  task  and  attempts  to  look  at  the 
actual  content  within  the  structure.  That  is,     assuming  that  such  constructs 
(strtuctures)  exist,  then  what  methods  can  be  used  to  determine  the  specific 
relationships  of  ideas  within  a  structure? 


DIFFERENT  TYPES  r   KNOWLEDGE  REPRESENTATION 

It  will  be  helpful  to  distinguish  between  different  assumptions  regarding  the 
representation  of  knowledge  in  memory.  This  will  affect  how  this  knowledge 
is  structured,  and  consequently  how  it  is  assessed.  There  is  not  complete 
agreement  in  current  cognitive  psychology  as  to  the  exact  definition  of  the 
terms  that  I  have  used,  but  I  have  grouped  some  common  constructs  together 
in  an  attempt  to  be  parsimonious. 


Propositions/semantic    memory.  This  type  of  representation  claims  that 
knowledge  is  stored  in  a  way  that  is  not  a  direct  representation  of  the  outside 
world.  Rather,  knowledge  is  abstracted  and  stored  in  the  form  of  propositions. 
These  are  relationships  between  subjects  and  predicates.  Schank  and  Abelson 
(1977)  have  expressed  the  idea  that  there  are  a  limited  number  of  relationships 
that  can  be  represented  generically,  in  their  Conceptual  Dependency  theory. 
Proposition  representations  are  assumed  to  be  declarative  knowledge,  that  is, 
knowledge  of  "what"  as  opposed  to  knowledge  of  "how."  (E.  Gagne,  1985).  This 
type  of  knowledge  would  equate  with  "verbal  information"  in  R.M.  Gagne's 
(1979)  taxonomy  and  with  the  "remember"  level  in  Merriirs  (1983)  Component 
Display  Theory. 

Productions.  When  knowledge  can  be  applied,  it  can  be  thought  of  as  existing 
in  a  production  representation.  This  is  a  series  of  "if/then"  statements  that 
allows  the  subject  to  react  to  the  outside  world.  Implicit  in  this  representation 
form  is  the  Mea  that  an  if/then  statement  can  be  linked  to  othei  if/then 
statements  creating  a  production  system  (E.  Gagne,  1985).  A  production  system 
would  underlie  all  complex  forms  of  performance,  both  cognitive  and 
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psychomotor.  This  type  of  knowledge  would  roughly  equate  with  "intellectual 
skills"  and  "motor  skills"  in  R.M.  Gagne's  (1979)  taxonomy  and  the  "use"  and 
"find"  levels  in  Merriirs  (1983)  Component  Display  Theory. 

Schemata/scripts,  Although  the  most  ambiguous  of  the  representation  forms,  a 
schema  or  a  script  can  generally  be  thought  of  as  knowledge  of  a  prototypical 
situation.  The  situation  activates  an  existing  script  or  schema  in  which  there 
are  default  values  for  the  most  typical  variables  that  will  be  encountered  in 
the  situation.  The  learner  can  go  beyond  the  information  given  by  using  these 
default  values  for  both  comprehension  and  creating  inferences.  Schemata  and 
scripts  are  essential  in  the  comprehension  process,  because  they  set  up 
expectations  of  the  environment  regarding  what  to  expect  in  a  situation.  More 
central  actions  may  be  more  easily  recalled  and  can  be  used  as  guides  for  top- 
down  processing  (Galambos,  1986).  It  would  seem  that  a  schema  or  a  script 
could  be  thought  of  as  an  integrated  system  of  propositions  and  productions 
since  both  need  to  interact  in  controlling  perception,  understanding,  and 
behavior. 

Images/episodic    memory/iconic    memory/ spatial    knowledge.  There  are 
similarities  and  differences  between  these  ideas  of  representation  forms. 
Essentially,  each  is  based  on  the  idea  that  knowledge  may  be  stored  in  a  literal 
**picture"  form,  as  opposed  to  an  abstracted  representation  such  as  a 
proposition. 


KNOWLEDGE  STRUCTURE 

This  section  on  assessing  knowledge  structure  briefly  details  several  methods 
for  assessing  structure   assuming   a  proposition/semantic   representation,  a 
schema  based  representation,  a  production   representation,   and  an 
iconic/episodic  representation.  As  was  previously  mentioned,  note  that  some 
of  these  techniques  are  concerned  with  proving  the  existence  of  the  construct 
while  others  are  used  to  assess  the  current  content  of  the  structure. 


SEMANTIC/PROPOSITION  REPRESENTATION 

These  tasks  and  tests  do  not  reveal  the  specific  contents  of  the  propositions  in 
memory.  Rather,  they  seek  to  confirm  the  existence  of  this  type  of 
representation. 


Reading  time  re^r^-This  basic  test  is  used  to  support  the  idea  of  propositions.  A 
sentence  is  broken  in  propositions  by  the  experimenter.  The  subject^s  reading 
time  is  measured.  Reading  time  correlates  strongly  with  number  of 
propositions  included  in  each  sentence,  because  it  is  assumed  that  it  takes  time 
to  process  each  different  proposition.  (Kintsch  &  Keenan,  1973,  cited  in 
Robertson,  1986) 

Semantic  recall  re^r^-Subjects  are  given  a  paragraph  to  read.  They  are  later 
shown  sentences  which  are  similar  to  some  of  the  sentences  contained  in  the 
original  paragraph.  The  fact  that  subjects  will  often  believe  that  they  have 
seen  a  sentence  that  is  similar  in  meaning— but  not  actually  incurred  in  the 
paragraph-supports  the  idea  that  information  may  be  stored  in  proposition 
fo  rm. 
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Hierarchical  timing  tests-li  was  assumed  by  Collins  and  Quillian  (1969,  cited  in 
Norman,  1976)  that  learners  structure  knowledge  of  a  concept  in  a 
hierarchical  manner,  with  essential  features  at  the  higher  levels.  Lower 
levels  contain  only  features  that  separate  the  item  from  other  items  at  the 
same  level.  This  was  measured  with  timing  tests*  assuming  that  it  took  more 
time  to  process  comparisons  between  items  at  different  levels  than  it  would 
take  to  compare  parallel  items  in  one  of  the  branches  of  the  hierarchy, 
because  of  the  processing  time  needed  to  search  through  the  hierarchy  (E. 
Gagne,  1985).  Although  continued  research  evidence  disproved  much  of  this 
theory,  the  timing  technique  may  still  have  value  (see  generating  examples, 
below). 


SCHEMATA/SCRIPTS/FRAMES  AS  REPRESENTATIONS 

These  tests  are  generally  based  on  the  assumption  of  semantic  proximity,  that 
is,  that  related  ideas  arc  stored  close  to  each  other  in  cognitive  structure 
(Nagy,  1983).  Most  of  these  assessment  procedures  deal  with  integrated 
propositions,   and  occasionally  propositions  integrated  with  production 
systems. 


Pattern   nofej— Subjects  are  given  a  word  at  the  center  of  a  blank  page  of  paper 
and  asked  to  free  associate  other  words  w^ich  are  then  linked  together. 
However,  the  student-created  linkages  do  not  specify  the  relationships 
between  the  words.  Pattern  notes  are  scored  by  counting  the  number  of  links 
between  words,  and  creating  an  intercorrelation  matrix.  This  test  has 
concurrent  validity  with  word  association  tests  (Jonassesn,  1987). 

Word  association  tests-  There  arc  two  types:  (1)  Free  word  association:  subjects 
arc  given  a  word  and  asked  to  Hst  the  first  word  that  comes  to  mind  when 
thinking  of  the  first  word.  Below  it,  the  second  word  that  comes  to  mind  is 
listed,  etc.  Chaining  is  not  allowed,  and  words  are  not  to  be  repeated.  This  list  is 
then  correlated  with  the  content  structure  (Preece,  1976).  (2)  Controlled  word 
association:  subjects  are  given  lists  of  words  and  asked  to  rank  order  them 
according  to  which  words  are  most  related  in  meaning.  (Jonassen,  1987). 

Tree   construction   tasks—  Subjects  are  required  to  construct  a  linear  graph  in 
which  the  intersections  are  words  and  the  connecting  lines  are  the 
relationships.  However,  structures  may  be  constrained  by  this  type  of  pattern. 
(Jonassesn,  1987) 

Concept  mflpj-Subjecis  arc  asked  to  link  concepts  that  relate  to  other  concepts 
in  a  hierarchical  pattern.  The  links  (representing  the  relationships  between 
concepts)  are  then  labeled.  It  is  assumed  that  this  is  a  "rubber"  hierarchy,  that 
is,  that  the  sequence  of  connected  concepts  could  be  subsumed  in  several 
hierarchies  of  equal  validity.  Nowack  and  Gowan  (1984)  have  devised  a  scoring 
system  in  which  concept  maps  can  be  compared  to  content  structure. 

Membership  decision   tasks—This  type  of  task  attempts  to  determine  the  typical 
actions  within  a  script.  "Subjects  are  timed  as  they  decide  whether  an  action  is 
involved  in  performing  a  given  activity"  (Galambos,  1986).  Subjects  are  first 
given  an  action  phrase  followed  by  an  activity  phrase.  The  subject  must 
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choose  whether  or  not  the  action  is  a  component  of  the  activity.  This  measures 
the  centrality  of  the  action,  that  is,  how  important  the  action  is  to  the  activity. 

Generating    examp lesSuhjccis  arc  given  a  category  and  asked  to  generate 
members  or  instances  of  the  category.  The  "production  frequency"     >^  speed 
and  Cider  of  generation  of  the  instances)  determines  how  closely  ;  distance 
is  related  to  the  supcrordinate  category.    As  related  to  Collins  and  Lo  us  (1975) 
spreading  activation  theory,  one  can  determine  that,  "the  greater  the  semantic 
relatedness  of  an  instance  and  a  category,  the  faster  tb^  decision  that  the 
instance  is  a  member  of  the  category."  (Eyscnck,  1987). 

Conceptual   models—These  are  essentially  concept  maps  of  content  structure  as 
compared  to  cognitive  structure.  These  models  contain  node  links 
representing  concepts,  and  the  linkages  are  labeled,  for  example,  "is  a 
consequence  of;  is  one  feature  of,  etc."  (Neill,  1970  cited  in  Romiszowski,  1981). 
Although  not  used  for  assessing  cognitive  structure,  this  technique  could 
easily  be  adapted. 

Concept   typicality   ratings-ln  this  task,  subjects  are  given  a  supcrordinate 
category,  and  then  asked  to  rate  the  typicality  of  different  given  instances  of 
the  category.  It  is  assumed  that  more  typical  instances  are  more  highly 
relatcd-and  in  closer  semantic  proximity  in  structure—than  less  typical 
instances  (Eyscnck,  1987). 


PRODUCTION  REPRESENTATION 

A  production  is  closely  related  to  the  idea  of  information  processing,  and  can 
also  be  thought  of  as  the  "executive  subroutine**  that  controls  psychomotor 
activities.  A  production  is  said  to  underlie  all  skilled  performance,  be  it 
cognitive  or  physical. 

Subtraction   technique  for  assessing  processing—These  techniques  arc  used  to 
measure  the  time  needed  to  complete  various  tasks.  It  is  assumed  that  if  the 
processing  takes  place  as  theorized,  that  this  will  correlate  with  the  time 
needed  to  complete  the  mental  task.  By  varying  only  one  element  of  the  task, 
and  consequently  determining  a  change  in  processing  time,  the  experimenter 
infers  that  processing  took  place  as  theorized.  This  type  of  technique  has  been 
used  to  study  information  processing  models  of  cognition  and  perception  (E. 
Gagne,  1985) 

Actual  taskS"Bec?iUse  it  is  assumed  that  a  production  system  underlies  and 
controls  skilled  automatic  performance,  tfie  absence  of  the  skill,  would  point  to 
an  incomplete  production  system.  According  to  R.M.  Gagne  (1979),  to  test  for 
intellectual  skills,  the  subject  should  be  given  novel  instances  of  concepts  to 
classify,  or  novel  situations  in  which  to  apply  a  principle.  Since  production 
systems  underlie  these  skills,  if  the  subject  is  able  to  do  so,  then  it  is  assumed 
that  there  is  a  production  system  in  place  that  supports  such  action. 

Think'Aloud   tasks— In  these  tasks,  the  subject  is  asked  to  speak  aloud  his  or  her 
thoughts  as  he  or  she  solves  a  problem  or  carries  out  an  activity.  The 
statements  should  be  roughly  parallel  to  the  **code"  of  the  production  systems, 
and  should  follow  the  same  logic.  This  was  used  by  Gordon  (1961)  in  his 
attempts  to  map  out  creative  processes  in  invento  3.  Critics  of  the  approach  say 
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that  speaking  aloud  may  interfere  with  the  actual  production  system,  by 
making  the  subject  self-conscious  of  otherwise  unconscious  mental  activities. 

Diagnostic  tests-By  carefully  creating  tests  that  measure  the  components  of  a 
process  (rather  than  just  Ihe  outcome)  the  production  systems  that  are  used  by 
the  subject  can  be  inferred.  This  type  of  test  is  common  in  mathematics,  when 
students  are  asked  to  "show  your  work."  The  teacher  can  then  reason  that 
because  cert^iin  steps  were  omitted  oj  incorrect,  that  these  productions  (or 
links  to  these  productions  within  the  overall  production  system)  are  faulty. 


EPISODICIICONIC/SPATIAL  REPRESENTATION 

Scanning   time-U  knowledge  is  structured  in  a  pictorial  form,  then  it  should 
take  more  time  to  scan  a  complex  image  than  it  would  to  scan  a  more  simple 
image.  This  type  of  test  asks  the  student  to  recall  a  particular  image  (for 
example,  how  many  panes  of  glass  there  are  in  your  bedroom  windows).  The 
time  that  it  takes  to  answer  is  highly  correlated  with  the  complexity  of  the 
image  (number  of  panes  of  glass),  therefore  supporting  the  idea  that  the 
subject  actually  "looked"  at  an  image  that  was  stored  in  memory  and  actually 
counted  the  panes  of  glass.  (E.  Gagne,  1985). 

Cognitive   maps—li  is  assumed  that  cognitive  maps  are  independent  of 
language  and  semantic  representation  because  moukeys  and  white  rats  also 
have  them  (Eysenck,  1987).  Tolman  first  measured  the  existerxe  of  these 
structures  by  placing  animals  in  mazes,  allowing  them  to  mn  free,  then 
placing  rewards  in  various  places  while  blocking  off  certain  avenues 
(LeFrancois,  1982).  Because  the  animals  still  chose  the  shortest  path,  Tolman 
reasoned  that  they  used  "maps'*  to  find  their  way  instead  of  stimulus-response 
chains  or  sequences. 

Human   mazes— To  determine  the  subject's  spatial  abilities  and  sense  of 
direction,  subjects  are  put  through  a  disorienting  situation  and  then  asked  to 
point  to  a  direction  or  object.  In  a  1977  study  by  Kozlowski  and  Bryant  (cited  in 
Smyth,  Morris,  Lvy,  and  Ellis,  1987)  subjects  were  put  though  a  maze  consisting 
of  tunnels  beneath  a  university.  The  students  with  better  sense  of  direction 
were  able  to  improve  their  ability  to  indicate  the  starting  point  over  several 
trials.  Those  with  weaker  sense  of  direction  are  unable  to  improve.  Witkin*s 
field  dependent/field  independent  aptitude  was  assessed  an  an  attempt  to 
determine  why  WWII  flyers  sometimes  got  lost  in  the  clouds.  His  earliest  test 
used  a  movable  chair  and  movable  horizon  (room)  to  determine  whether 
subjects  were  able  to  spatially  orient  themselves  to  true  vertical.  Later  tests 
expanded  his  original  concept  (Witkin,  Moore,  Goodenough,  Cox,  1976). 

Mental  rotation  tasks-A  subject  is  given  two  pictures  of  an  object,  and  asked  to 
determine  whether  or  not  the  first  object  can  be  rotated  to  look  like  the  second 
object.  This  can  be  done  with  two  or  three  dimensional  pictures.  Because  it 
takes  longer  to  answer  the  question  the  further  the  actual  object  would  have  to 
have  been  rotated,  one  determines  that  a  form  of  spatial  or  pictorial  knowledge 
structure  exists.  (E.  Gagne,  1985;  Smyth,  Morris,  Lvy,  and  Ellis,  1987). 
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Abstract 

This  paper  examines  the  possibility  of  a  Kuhnian  paradigm  in  research  on 
educational  communications  and  technology.  It  discusses  Kuhnian  metaphor 
and  its  attraction  as  a  viewfinder.  It  turns  to  anthropology  for  explanation  of 
how  beliefs  and  behaviors  become  social  fact.  Then  it  takes  a  critical  look  at 
usage  of  this  met^hor  in  the  field  as  a  social  text.  It  concludes  by  discussing 
the  schism  between  researchers  and  practitioners  in  the  ways  they  know  the 
world. 
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Empirical  Fact  Or  Social  Fact? 


Kuhn's  metaphorical  book  The  Structure  of  SdentiSc  Revolutions  (1970) 
has  itself  become  a  metaphor  smce  its  first  publication  in  1962.  Kuhn  iised 
citation  analysis,  an  empirical  historical  method,  for  building  his  model  of 
how  empiric^  thought  develops  in  physics  and  astnmomy.  There  is  irony  in 
empiricists  from  other  disciplines  borrowing  his  noodel  for  they  do  not  apply 
his  methodology.  They  account  for  their  own  fields  with  rational  tools  they 
would  not  normally  allow  in  their  wo^  due  to  lack  of  empirical  rigor. 
Invoking  the  authority  of  the  Gospel  of  Saint  Thomas  Kuhn  is  no  rarity  in  the 
second  half  of  the  twentieth  centiiry.  The  scripture  of  revolution  has  been 
adopted  as  the  text  for  hisiness  as  usual.  The  metaphorical  structure  has 
diffused  from  describing  scientific  research  to  the  point  that  it  applies  to 
almost  any  perceived  state  or  desired  change,  including  improvements  in 
computer  operating  systems.  Perhaps  the  dying  metaphor  will  receive  burial 
when  it  becomes  a  household  word.  I  imagine  the  day  when  advertising 
copywriters  use  paradigm  shiRas  a  replacement  for  new  improved  classic 
taste. 


Scholariv  Knowledge  About  Educational  Communications  And  Technology 

The  twentieth  century  disenchantment  with  epistemology,  with 
metaphysics,  and  with  philosophy  as  the  guiding  light  of  the  sciences  has 
resulted  in  confusion.  Lx)gical  empiricism  claims  objectivity  but  fails  any  test 
of  neutrality  because  science  in  action  is  a  sociopolitical  business  with 
sociopolitical  aims.  Social  scientists  intend  to  understand  social  facts  but 
seldom  know  why  tliose  facts  have  been  selected  for  study  or,  indeed,  how 
their  discipline  creates  social  facts.  Scholars  of  education  borrow  theory  and 
methods  from  these  researchers.  Confined  by  social  discontinuities,  the  field 
of  educational  communications  and  technology  cuts  across  education. 

Social  Facts 

There  are  two  kinds  of  human  action,  writes  Suchman  in  describing  the 
laige  contribution  of  G.  H.  Mead  to  her  theory  of  situated  action  (1987).  One 
is  actually  acting  and  the  other  is  derivative  in  that  it  plans  for  the  future  and 
retrospectively  accounts  for  the  past.  These  projections  or  reconstructions 
become  the  social  facts.  The  social  text  is  not  mere  discourse  but  becomes 
part  of  the  aciion.  Rabinow  looks  specifically  at  scholariy  representations  as 
social  facts  (1986)  and  this  is  a  recurrent  theme  in  contemporary  cultural 
anthropology  (Clifford  &  Marcus,  1986;  Geeitz,  1973,  1983,  1988).  These 
social  facts  are  interpretations.  In  contrast,  cognitive  scientists  should  have 
difficulty  with  this  concept  because  they  accept  words  as  equivalent  to  the 
objects  they  denote.  Winograd  and  Flores  (1987)  have  given  the  name  naive 
realism  to  this  commitment  to  rationalism. 


Empirical  Mythology 
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How  Do  Beliefs  And  Behaviors  Become  Social  Fact? 

Harris  has  proposed  a  theory  of  cultural  evolutioQ  that  accounts  for  how 
beliefs  and  behaviors  are  formed  in  response  to  environmental  pressures 
(1968,  1933).  There  is  no  good  reason  to  expect  that  intelligent  life  grows  and 
reacts  in  the  same  way  that  mold  cm  a  crust  of  bread  grows  and  reacts.  Harris' 
theory  of  cultiual  materialism  explains  that  beliefs  and  behaviors  are  shaped 
by  the  fundamental  issues  of  food  supply  and  population  growth  (1985).  He 
has  also  extrapolated  his  theory  to  hyperim'  istrial  American  life  (1987).  His 
theory  will  be  used  later  in  this  paper  to  show  that  claiming  a  paradigm  of 
educational  communications  and  technology  performs  a  social  function. 

An  Empirical  Factfinding  Mission  Towards  Prediction 

Ever  since  the  \isual  education  movement  emerged  in  the  i920s  due  to  the 
frequent  installation  of  motion  picture  projectors  in  schools,  researchers  have 
been  concerned  with  justifying  the  pretheoretical  assumption  that  technology 
can  improve  instiuctional  effectiveness  and  efficiency.  They  act  in  accord 
with  the  scientific  management  reforms  described  by  Callahan  (1962)  and 
Cuban  (1986,  p.  86-88).  In  a  landmark  investigation  funded  by  a  $10,000 
grant.  Freeman  supervised  the  13  studies  reported  by  his  assistants  (1924). 
Their  results  suggested  that  visuals  convey  a  different  kind  of  information 
from  that  conveyed  by  language.  Under  the  sponsorship  of  the  Eastman 
Kodak  Company,  he  disregarded  his  own  conclusions  and  conducted  a  widely 
publicized  media  comparison  experiment  (Wood  &  Freeman,  1929).  Over  a 
period  of  12  weeks,  20  films  were  shown  to  1 1,000  children  in  12  cities. 
Wood  and  Freeman  had  the  80,000  test  booklets  scored  and  found  that  film 
students  provided  more  concrete  answers  to  descriptive  questions  while  giving 
answers  to  abstract  questions  that  were  equal  to  those  given  by  the  control 
group.  This  quantification  justified  educational  motion  pictures  as  a  panacea 
and  fitted  in  with  the  agenda  of  Freeman's  dean,  Charies  Judd-see  ClifTord 
and  Guthrie  (1988,  p.  78)-for  establishing  a  science  of  education. 

The  tr^ition  of  experimental  psychology  in  our  field  is  the  basis  for 
c:laiming  a  cognitive  paradigm.  This  reductionist  line  of  research  continued 
for  over  65  years  with  spurts  of  excitement  as  each  new  medium  appeared,  see 
Dale,  Dunn,  Hoban  Jr.  and  Schneider  (1937),  McClusky  (1950),  Saettler 
(1968)  and  Wilkinson  (1980).  Li  1970,  Heinich  announced  in  his  monograph 
Technology  and  tiie  Management  of  Instruction  the  replacement  of  a 
behavioristic  paradigm  based  on  objective,  observable  behavior  with  a 
paradigm  for  learning  theory.  Clark  and  Salomon  (1986)  document  this  as  a 
reality  hit  unlike  Heinich  they  do  not  refer  to  Kuhn.  Clark  and  Sugnie 
reviewed  the  last  decade  of  research  on  instnictional  media  and  identified  four 
types  of  research:  behavioral,  cognitive,  attitudinal  and  economic  (1988>. 
Their  narrow  perception  of  paradigm  shifi  in  educational  media  centers  on 
"the  new  cognitive  theories  of  leaming  from  instmction"*  with  achievement  as 
the  dependent  variable.  Winn  expresses  a  strong  belief  in  leaming  as  the 
"bottom  line*'  (1988,  1989)  and  so  does  Clark  (1989).  By  leaming  they  mean 
the  transfonnation  of  individual  minds  and  not  the  social  process.  Ironically, 


714 

ERIC  712 


Empirical  Fact  Or  Social  Fact? 


5 

those  who  profess  the  cognitive  paradigm  are  also  charged  with  giving  stamps 
of  social  approval  to  the  practitioners  who  design  and  administer  instmction. 
No  wonder  Wiim  writes,  **...there  has  always  been  opposition  within  the  field 
to  research  and  theory  that  has  at  times  bordered  on  hostility (1988,  p.  808). 

Understanding  The  Transmission  Of  Culture 

A  crack  is  showing  in  the  cognitive  tgg  but  no  chick  is  ready  to  hatch.  In 
contrast  to  the  traditional  reductionist  research,  cognitive  scientists  themselves 
are  objecting  that  the  interactions  which  make  up  knowledge  communication 
cannot  be  isolated  from  situational  dimensions  (Suchman,  1987)  and  cultural 
fabric  (Lave,  1988).  From  this  viewpoint,  sociocultural  research  apait  from 
the  mainstream  represents  a  strength  of  educational  communications  and 
technology's  position  as  an  applied  field. 

The  scholars  investigating  this  dimension  recognize  that  much  more  is  at 
stake  in  educational  communications  and  technology  than  achievement,  which 
is  only  the  ostensible  purpose  of  schooling.  Their  work  is  closer  to  practice 
because  it  represents  the  need  to  solve  real  world  problems.  Driven  by 
sociocultural  research  issues,  their  investigations  cross  the  breadth  of  theories 
and  methods  of  the  social  sciences  and  the  humanities.  For  example,  Belland, 
Duncan  &  Deckman  (1985),  Becker,  (1986),  Hynka  (1989),  Januszewski 
(1988),  Koetring  (1988),  Muffoletto  (1987)  and  Taylor  &  Swartz  (1988)  are 
concerned  about  critically  understanding  cuitund  reproduction.  They  write  in 
what  Husen  (1988)  idenrifies  as  educations'  other  research  paradigm:  the 
humanistic  way  of  knowing,  as  opposed  to  neopositivism/logical  empiricism. 

Kuhn's  Gospel 

A  Stained  Glass  Window 

Kuhn  created  his  model  to  explain  the  results  of  his  bibliographic  research 
on  the  history  of  science  (1970).  His  views  opposed  Popper's  principle  of 
falsi fiability  that  had  been  accepted  for  three  decades  (Popper,  1968).  In  the 
Popperian  tradition  the  research  strategy  competition  in  science  is  far  from 
futile.  It  is  thought  advantageous  because  the  credibility  of  steadystate 
knowledge  rests  not  on  dogma  but  on  refinement  and  replacement  by  more 
powerful  theories  and  closer  approximations  of  truth.  Popper,  himself,  had 
been  a  radical  in  asserting  the  superiority  of  empirical  observation  over  the 
acceptance  of  scientific  theories  by  agreement  between  authorities.  Kuhn 
questioned  Popper's  textbook  explanation  of  continual,  rational  progress. 

Kuhn's  scripture  contains  three  central  metaphors:   paradigm,  anomaly 
and  revolution.  The  pamdigm  is  an  accepted  set  of  mles  for  knowing  about 
and  conducting  normal  science.  The  anomaly  is  the  exception  that  stimulates 
new  explanations  that  cannot  be  ignored.  The  revolution  is  the  emergence  of 
a  new  paradigm. 

Kuhn  identified  three  normal  foci  of  factual  scientific  investigation  (1970, 
p.  25-30).  They  sometimes  overlap  and  I  have  added  the  parenthetical  phrases 
to  improve  the  distinctions: 
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Determination  of  significant  fact  (applying  research  strategics). 
Paradigmatic  facts  are  developed  for  solving  paradigmatic  problems. 
Measurement  occurs  with  increasingly  refined  {^paratus  and  methods.  Some 
researchers  receive  more  recognition  for  developing  research  tools  than  for 
what  they  find. 

Matching  facts  with  theorv  (addressing  research  issues). 
Increasing  the  match  between  theory  and  r^;^e  comes  from  arguments  and/or 
factfinding  demonstrations  rooted  in  the  real  world:  **The  existence  of  the 
paradigm  sets  the  problem  to  be  solved." 

Articulating  theorv. 
A  paradigm  is  articulated  by  looking  for  universal  constraints,  qimntitative 
laws  and  experimer'3,  in  a  more  qualitative  sense,  that  elucidate  a 
phenomenon  and  relieve  ambiguous  interpretations. 

This  is  not  falsification  (Kuhn,  1970,  p.  77-80).  New  basic  theories  appear 
in  the  short  revolutionary  period  of  extraordinary  science  but  new  paradigms 
are  incommensurable  with  previous  paradigms.  In  the  long  periods  of  normal 
science^  the  reigning  paradigm  restricts  researchers  to  puzzle  solving  science 
in  which  assumptions  are  not  questioned. 

Kirfm  (1970)  further  questioned  modem  rationality  by  doubting  the 
neutrality  of  investigators  and  suggested  the  importance  of  considering 
sociological  and  individual  psychological  aspects.  He  stated,  "Lifelong 
resistance,  particularly  from  those  whose  productive  careers  have  committed 
them  to  an  older  tradition  of  normal  science,  is  not  a  violation  of  scientific 
standards  but  an  index  to  the  nature  of  scientific  research  itself."*  (p.  151). 
This  relativistic  approach  led  to  debates  with  other  philosophers  and  historians 
of  science  including  Popper,  who  defended  the  n^trality  of  scientists,  and 
Feyeiabend,  who  welcomed  subjectivity  and  opened  the  back  door  to 
mysticism.  Further  debate  with  Lalratos  chased  empiricism  into  a  comer  and 
Kuhn  demonstrated  the  importance  of  scientific  consensus  in  determining 
hanj  facts  (Lakatos  &  Musgrave,  1970). 

Over  the  years  Kuhn's  bibliographic  analyses  increased  in  particularity 
because  each  revolution  requires  a  unique  explarmtion.  His  original  metaphors 
did  not  fit  all  situations.  For  instance,  he  discovered  differing  paradigms  for 
chemistry  and  physics  that  described  helium  either  as  an  atom  or  » 
molecule  (1970,  p.  50-51)  and  he  drew  parallels  from  science  to  art,  from 
theories  lo  painting  styles  (1977,  p.  340-351).  With  icf'nemint,  KuhnV.  model 
was  reduced  in  scale  from  paradigms  in  conflict  to  theories  in  conflict  There 
is  an  admitted  lack  of  generality^  partly  in  response  to  close  examinations  of 
the  issues,  see  the  table  illustrating  Stegmuller^s  chapter  on  ""Kuhnianism": 
Pseudoirrationalism  and  Pseudorelativism,  (1976,  p.  241-246).  Barnes  shows 
that  Kuhn's  ideas  have  become  progressively  more  conformist,  conservative 
and  more  supportive  of  the  scientific  establishment  (1983).  Paradigmhds  lost 
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its  revolutionary  fire:  it  may  mean  no  more  than  Weltanschauung  as  a 
metaphysical,  epistemological  and  methodological  perspective. 

The  Kuhnian  Scriptures  And  Social  Science 


In  examining  the  relationship  of  Kuhn  to  our  field,  it  is  first  necessary  to 
review  Kuhn's  influence  on  the  social  sciences,  the  source  of  our  theories  and 
research  meUiods.  Kuhn  ventured  ""...it  remains  an  open  question  what  parts 
of  social  science  have  yet  acquired  such  paradigms  at  all.  History  suggests 
that  the  road  to  a  firm  research  coiisensus  is  extraordinarily  arduous."*  (1970, 


What  attracted  Barnes  to  write  on  TS.  Kuhn  and  the  Sodal  Sciences 
(1984),  despite  Kuhn  avoiding  extrapolation  of  his  ideas  to  the  social  sciences, 
was  Kuhn's  central  argument  that  more  is  going  on  in  science  than  an 
academic  competition  of  ideas.  Barnes  describes  the  dangers  of  Whig  history, 
of  viewing  the  past  as  a  reflection  of  the  present,  and  of  writing  textbooks  that 
present  only  facts  that  support  current  understandings.  He  also  uses  the 
phrase  "intellectual  laziness"  (p.  120)  to  show  that  he  does  not  endorse  the 
dogmatic  acceptance  of  Kuhn's  model  as  an  after  the  fact  e;q}lanation  in 
sociology,  economics  or  psychology. 
Kuhn  (1970)  cautioned: 

"The  members  of  all  scientific  communities,  including  the  schools  of  the 
"pre-paradigm"  period,  share  the  sorts  of  elements  which  I  have  collectively 
labeled  "a  paradigm."  What  changes  with  the  transition  to  maturity  is  not  the 
presence  of  a  paradigm  but  rather  its  nature.  Only  afler  the  change  is  normal 
puzzle-solving  research  possible."  (p.  179), 

This  is  not  enough.  Just  as  eariy  psychological  researchers  had  physics 
envy  (Gould,  1986,  p.  262-263)  and  some  social  sciences  have  been  the 
subject  of  debates  over  whether  they  arc  really  scientific  or  not  (Kuhn,  1970, 
p.  160),  the  acknowledgement  of  a  paradigm  is  socially  desirable  as  a  sign  of  a 
discipline's  intellectual  adulthood.  Cogniti^^e  psychologists  claim  that 
understanding  Kuhnian  paradigm  shifts  in  their  field  is  central  to 
understanding  cognitive  psychology  (Lachman,  Lachman  &  Butterfield, 
1979).  In  linguistics  there  was  a  I^hnian  diHUsion  of  ideas  through  a  new 
generation  (Gardner,  1986,  p.  209).  Even  paraj^ychologists  are  attracted  by 
the  orthodoxy  of  Kuhnian  metaphors  (Barnes,  1983,  p.  90-93;  Radner  & 
Radner,  1982,  p.  62-67). 

Mitchell  (1979),  an  English  professor,  could  explain  why  no  area  wants  to 
be  regarded  as  preparadigmatic: 

"We  can  always  tell  which  of  two  crafts  outranks  the  other  by  looking  at 
its  .  .icon.  Jargon  only  mns  downhill.  You  will  notice  that  although 
educators  have  borrowed  "input"  from  the  computer  people,  the  computer 
people  have  felt  no  need  to  borrow  ^'behavioral  objectives"  or  "preassessment" 
from  the  educators."  (p.  106) 


p.  15). 
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The  Cognitive  Paradigm  Metaphor  As  A  Sodai  Text 

As  the  author  of  a  classic  work,  Ruhn  has  endmed  the  fete  of  classic 
authors  in  that  his  scripture  has  been  more  often  cited  than  read  (Adams  & 
Searle,  1986,  p.  381).  This  can  be  understood  in  the  same  way  Johnson-Laird 
(1988,  p.  21)  debunks  Levi-Strauss  and  Piaget:  "A  theory  may  be  so  rich  in 
descriptive  possibiUties  that  it  can  be  made  to  flt  any  data.""  In  other  words, 
"Even  things  that  are  tnie  can  be  proved,''  (Wilde,  1891/1976,  p.  5). 

The  cognitive  paradigm  in  educational  ccmununications  and  technology  is 
a  social  text  that  encourages  cooperation  in  the  joint  productive  effort  like  the 
boost  in  agricultural  production  from  a  Kwakiutl  chief  stdistributing  wealth  at 
a  potlatch  (Harris,  1974)  or  the  feast  of  a  Solcnnon  Islands  mumi  (Harris, 
1977).  Having  a  paradigm  is  an  indicaticm  of  being  established.  The 
transition  from  preparadigni  state  to  postparadigm  state  is  videly  perceived  as 
the  passage  of  pubeity  for  any  discipline.  From  the  viewpoint  of  cultural 
evolution,  claiming  a  paradigm  has  adaptive  value.  It  helps  people  obtain  and 
maintain  employment.  If  ihey  say  they  are  believers  they  incjease  their 
chances  of  survival  in  the  job  market.  Conflict  is  discouraged  h^jcause  it 
decreases  material  growth*  Allowing  people  to  examine  what  they  do  and 
why  they  do  it  causes  schisms  and  these  reduce  the  centralized  po  wer. 
Grandiose  claims  impress  other  big  men^  such  as  granting  agencies  and  they 
reassure  school  district  superintendents  and  corporate  directors  of  instnictional 
systems  that  someone  really  knows  what  is  going  on.  Everything  appears 
under  control  and  the  scholars  who  claim  a  cognitive  paradigm  on  the  elite 
campuses  promise  to  bend  their  research  efforts  to  everyone's  benefit. 

Another  interpretation  fits  just  as  well  because  the  same  social  mechanism 
could  exist  without  actual  paradigmatic  consensus.  If  there  is  a  Kuhnian 
paradigm  in  educational  communications  and  technology',  why  does  it  mimic 
what  Apple  and  Weiss  (1983,  p.  3)  call  the  dwninant,  achievement  ipeasuring 
tradition  of  educational  research?  In  Kuhn's  scripture,  preparadigrnatic 
research  is  characterized  by  the  atheoretical  factfinding  characteristic  of 
prescientific  times  (1970,  p.  15-16).  Explanations  of  phenomena  are 
inadequate.  Data  are  too  dense  for  decoding.  Details  are  missed  which  are 
later  considered  important.  In  preparadigmatic  research,  technology  is  the 
name  given  to  solving  practical  problems  systematically.  Technology  mothers 
science. 

In  our  field  the  technique  is  the  systems  approach  to  instmctional 
management,  also  identified  by  Heinich  as  the  result  of  paradigm  shift  (1970). 
Today  the  technique  is  known  as  Instructional  Systems  Development  (ISD). 
The  field  has  become  more  complex  in  the  two  decades  since  Heinich's 
observation  and  choosing  betv;'een  prescriptions  is  increasingly  difficuU.  !n  a 
recent  study,  30  ISD  models  were  identified  and  compared  for  12  major  tasks 
(Taylor  &  Doughty,  1988).  Further,  Reigeluth  (1989)  warns  that  research  and 
development  may  be  out  of  touch  with  each  other. 


^  This  phrase  comes  from  anthropology. 
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Alternatives  To  The  Received  Research  Agenda 

From  the  cultural  materialist  point  of  view,  individuals  and  groups  can  still 
use  their  mtellect  to  choose  what  they  want  to  believe  and  what  they  want  to 
do.  Ifarris  thinks  that  despite  lack  of  evidence  of  this  in  the  past,  it  may  be  the 
only  hope  for  our  planet  in  the  face  of  the  ecological  emergency  (1987,  p. 
18 1-183).  In  our  field,  for  example,  we  mi^t  turn  aside  from  the  mechanical 
study  of  achievement  and  select  the  direction  of  Habermasian  morality: 

"^...education  should  function,  via  communicative  action,  to  help  us 
competently  reach  understanding  with  one  another  (the  cultural  fuixtion), 
fuifill  appropriate  societal  norms  (the  social  function),  and  develop  c'ir 
personalities  (the  socialization  function),  and  in  the  process,  learners  become 
involved  with  objective,  practical  and  emancipatory  forms  of  knowledge." 
(Nichols,  1989,  p.  9) 

As  a  further  example  of  an  alternative  to  measuring  achievement,  I  suggest 
looking  for  the  special  contributions  of  educational  media  in  giving  learners 
experiences  which  they  can  shape  for  themselves.  I  suggest  evading  the 
politics  of  schooling  by  finding  ways  cf  preventing  the  arbitrary  constraints  of 
time  and  place  from  interfering  in  individual  learning.  Also,  I  encourage 
educators  to  take  an  ethical  stance  and  accept  that  achievement  differences  are 
not  only  unimportant  but  they  are  to  be  expected  as  normal  and  healthy. 
Achievement  differences  among  the  normal  population  not  only  do  not  need 
to  he  overcome  but  they  should  not  be  overcome. 

The  Schism  Between  Researchers  And  Practitioners 

As  an  applied  field,  the  Kuhnian  paradigm  of  revolutionary  process  does 
not  fit  educational  communications  and  technology  and  neither  does  the 
falsificationist  myth  of  ordedy  progress.  The  description  of  the 
preparadigmatic  state  may  ni  equally  well  or  even  better.  Nevertheless, 
Kuhn's  contribution  in  identifying  the  sociopolitical  undeipinnings  of  science 
remains  and  there  are  material  pressures  for  claiming  a  cognitive  paradigm. 
That  claim  is  a  social  fact,  whether  it  is  tme  or  not. 

The  research  base  of  educational  communicadons  and  technology  is 
scientific  but  the  practice  is  not  the  practice  of  science.  Technological 
determinism  has  a  circular  logic:  Insi^  actional  science  resembles  medicine 
which  is  an  applied  science  (Reigeluth.  1984,  p.  20)  but  education  is  also 
claimed  as  an  art  (De  Landsheere,  1988,  p.  16).  The  splh  is  well  known  from 
C.  P.  Snow's  Tw>  Cultures  essay  (1959/1965)  and  his  semiauiobiographical 
novel  7^.  Si^an'h  (1935/1958).  Hlynka  argues  for  the  necessity  of  both 
approaches  and  both  metaphors  in  our  technological  field  (1984).  There 
remains  a  paradox  in  that  the  practice  of  educational  communications  and 
technology  is  not  the  practice  of  education.  Jonassen,  for  instance,  writes: 
"Educational  technology  is  the  science  of  instruction...^'  (1984,  p.  159). 

The  cognitive  research  paradigm  is  so  concerned  with  product  that  it  can 
have  no  meaningful  response  to  process  questions  such  as  Apple's  "Is  the  r 
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technology  part  of  the  solution  or  part  of  the  problem  in  education?**  (1988, 
p.  150).  For  example,  quantitative  tests  show  significant  difTerences  between 
students*  levels  of  computer  anxiety  (Glass  &  Kdg^t,  1988)  but  reliable 
measurements  do  not  support  the  validity  of  a  constmct  (Cherryholmes,1988). 
The  results  have  been  applied  without  regard  for  their  validity.  The 
consequences  of  computer  anxiety  research  reports  provide  evidence  that 
technology  is  not  neutral  and  that  science  can  be  used  to  dcnninate  people. 

The  interaction  between  the  design  of  things  and  the  design  of  instruction 
reveals  the  politics  of  the  received  research  agenda  in  educational 
communications  and  technology.  Norman  (1988,  p.  4043)  believes  that 
badly  designed  hardware  and  sofhvare  teach  people  to  be  helpless  when 
working  with  technological  items.  In  contrast  to  that  cognitive  engineering 
viewpoint  it  is  incongmous  that  the  computerphobia  programs  deliberately 
teach  people  they  are  helpless.  These  interventions  tell  people  they  are 
ineffective  and  inci^jable  and  that  they  need  special  instruction  to  get  over 
their  disability.  I  disagree  with  these  procedures  and  my  values  are  probably 
similar  to  the  ones  you  hold:  I  believe  that  things  need  fixing,  not  people.  My 
comment  is  that  the  act  of  teaching  teachers  and  students  that  they  are 
computer  anxious,  that  they  are  helpless,  that  they  need  remediation,  may  be 
an  oppressive  act.  Sociocultural  issues  such  as  these  need  to  be  explored  with 
research  based  on  sound,  theoretical  foundations  drawn  broadly  from  the 
social  sciences  and  the  humanities. 

Educational  coi^.imunications  and  technology  is  rooted  in  the  practice  of 
using  and  producing  educational  media  but  researchers  do  one  set  of  things 
and  practitioners  do  another  set  of  things.  Classroom  teachers,  school  media 
coordinators  and  instructional  system  developers  select  images  to  convey  the 
world  to  learners.  They  know  the  field  is  effective  because  educational  media 
employ  rhetoric.  The  field  is  poetic,  lyric,  dramatic,  epic  and  alive  with 
metaphors.  Researchers  investigate  to  write  scholariy  research  reports  but 
educational  cinematographcrs  investigate  to  create  films  and  videos.  The  way 
of  saying  something  changes  what  is  said.  In  practice,  far  beyond  the  dusty 
educational  psychology  journals  that  fill  shelf  after  shelf  of  university 
libraries,  we  have  an  artistic,  creative,  imaginative  field. 

Adams*  Philosophy  of  the  Dterary  Symbolic  {\9S3)  confronts  the  tension 
between  the  literary  and  the  scientific  ways  of  knowing:  "The  war  between 
poetry  and  philosophy  has  extended  from  before  Plato*s  time  into  our  own." 
(p.  389).  These  words  also  apply  to  educational  communications  and 
technology.  More  than  a  paradigm,  debates  between  researchers  and  between 
researchers  and  practitioners  ensure  conscious  decisions.  More  than  a 
paradigm,  the  field  needs  this  conflict  between  the  poetry  of  practice  and  the 
philosophy  of  research. 
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Introduction 

A  research  project,  utilizing  both  quantitative  and 
qualitative  methods,  was  designed  and  conducted  to  study 
the  distance  education  component  of  the  Zimbabwean  teacher 
education  program.     During  this  study,   in  which  the 
research  design  evolved  during  the  project,   a  model  combing 
three  theoretical  perspectives  on  education  was  developed 
by  the  author.     This  model,   the  Triad  Perspective  Model  of 
Distance  Education,   integrates  Systems  Theory,  Curriculum 
Development  Theory,   and  Adoption  and  Diffusion  of 
Innovations  Theory.     By  using  the  model  as  a  guide,  a 
comprehensive  description  of  the  program  was  created,  the 
strengths  and  weaknesses  of  the  program  were  identified, 
and  recommendations  for  program  improvement  were  presented. 

Zimbabwean  Teacher  Education 

Zimbabwe*  s  current  educational  structure  is  a  result 
of  the  shift  from  British  colonial  rule  to  independence  in 
April,   1980.     Prior  to  that  time,  public  education  was 
divided  along  racial  lines,   and  schools  were  strictly 
segregated.     The  educational  system  was  a  way  of 
"establishing  and  maintaining  the  social  and  economic 
differences  between  the  black  majority  and  the  white 
minority"     (Kadhani   ^  Riddell,   1981,  p. 58). 

Political  upheaval  eventually  brought  about  major 
changes  in  the  country's  educational  priorities.  Upon 
independence,   Zimbabwe  was  in  a  position  to  offer 
educational  opportunity  to  all  of  its  citizens,  regardless 
of  color.     Unfortunately,   the  radical  changes  brought  about 
by  a  new  political  philosophy  could  not  be  immediately 
assimilated  and  a  "crisis  of  expectation"  assaulted  the 
schools.     Enrollments  increased  dramatically,  challenging 
the  resourcefulness  and  ingenuity  of  Zimbabwean  educators. 
Between  1979  (pre-independence)   and  1984  there  was  a  252% 
increase  in  the  number  of  students  enrolled  in  primary 
schools.     The  expansion  at  the  secondary  level  was  even 
greater.     Between  1979  and  1984,   secondary  enrollments 
increased  by  638%   (Chivore,   1986) . 

The  number  of  professionally-qualified  teachers 
employed  in  the  Zimbabwean  educational  system  could  not 
begin  to  meet  the  need.     In  1985,   over  66%  of  the  secondary 
school  teachers  were  untrained  or  underqualif ied,   and  the 
situation  was  considered  "critical"  at  the  primary  level, 
as  well   (Chivore,   1986)  .     This  shortage  of  qualified 
teachers  led  to  major  changes  in  the  teacher  education 
program. 

Teacher  Education  j.n  Zimbabwe 

The  original ,  pre-independence  model  for  teacher 
education  was  quite  similar  to  the  traditional  American 


model.     Students  attended  a  specialized  Teachers'  College 
for  training,  of  which  there  are  now  fourteen  in  Zimbabwe. 
Teacher  education  consisted  of  a  three-year  campus-based 
program  that  included  periods  of  "teaching  practice" 
(similar  to  student  teaching)   with  increasing  classroom 
responsibilities  over  time,   under  the  direct  supervision  of 
a  full-time  classroom  teacher. 

This  model  was  altered  in  response  to  the  teacher 
shortage  that  occurred  after  independence  was  achieved. 
The  student's  responsibilities  while  on  teaching  practice 
(T.P.)   increased,  as  did  the  amount  of  time  spent  on  the 
T.P.  assignment.     The  current  model,  adopted  in  1988, 
includes  one  year  of  on-campus  instruction,   one  year  of 
teaching  practice   (during  which  the  teacher  education 
students  shoulder  the  responsibilities  of  full-time 
teaching),  and  the  final  year  back  on  campus. 

Another  model  of  teacher  education  was  created  within 
the  Zimbabwe  Integrated  National  Teacher  Education  Course 
(ZINTEC) .     Four  ZINTEC  colleges  were  built  shortly  after 
independence  specifically  to  alleviate  the  teacher  shortage 
crisis  by  putting  large  numbers  of  teachers  into  classrooms 
after  a  short,    intensive  training  period.     The  ZINTEC 
model,  as  it  now  exists,   is  a  four-year  program  with  two 
terms   (four  months  each)  of  on-campus  instruction,  followed 
by  eight  terms  of  T.P.,   concluding  with  two  terms  on 
campus . 

Distance  Education  and  the  Teacher  Education  Program 

Distance  education  was  incorporated  into  the  teacher 
education  program  so  that  students  on  teaching  practice 
assignments  could  continue  their  academic  progress.  The 
program  has  been  predominantly  print-based,   utilizing  self- 
contained  "modules"  for  instruction.     The  students  work 
from  these  learning  packages  and  return  assignments  when 
visiting  the  campus  for  seminars  or  at  the  end  of  their 
T.P .  assignment . 

The  modules  are  prepared  by  a  central  agency,  the 
Distance  Education  Centre,   with  assistance  from  teachers' 
college  faculty  who  are  released  on  "sabbatical"  to  act  as 
subject  matter  specialists.     Because  academic  resources  are 
often  scarce  in  the  outlying  rural  areas  where  many 
students  are  assigned  to  teaching  practice,   the  ] earning 
packages  contain  the  necessary  reading  materials  and 
emphasize  projects  that  can  be  completed  with  locally- 
available  supplies . 

Studying  the  Distance  Education  Program 
A  research  project  was  designed  with  the  purpose  of 
developing  a  narrative  description  of  the  Zimbabwean 
distance  education  program's  strengths  and  weaknesses. 
Recommendations  for  further  development  would  then  be  drawn 
from  these  results. 
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with  the  assistance  of  a  grant  from  the  United  States 
Information  Agency,  through  its  Teachers/Texts/Technology 
program,   two  trips  were  made  to  Zimbabwe  by  the  author  to 
observe  the  distance  education  program.     In  addition  to 
observation,   interviews  were  conducted  and  questionnaires 
distributed  to  assess  the  role  and  status  of  distance 
education  in  the  teacher  education  program. 

Triad  Perspecl- 1  vf:>  Model  of  Distanrfi  F.dunation 

As  in  many  qualitative  studies,   an  "emergent  research 
design"  began  to  evolve  as  the  project  progressed.  A 
model,  developed  from  three  theory  bases,  wa.   created  by 
the  author  to  facilitate  the  comprehensive  description  of 
the  distance  education  program.     These  theoretical 
perspectives  were  the  foundation  for  the  study  and  included 
Tyler's  Curriculum  Development  theory  (1949),  Banathy's 
Systems  theory   (1973),   and  Rogers'  theory  of  Adoption  and 
Diffusion  of  Innovations  (1983). 

The  core  ideas  within  each  of  the  three  theory  bases 
were  incorporated  into  an  integrated  model,  the  Triad 
Perspective  Model  of  Distance  Education   (TPMDE) .  (See 
Figure  1.)     Each  theory  base  represents  a  vertical  column 
within  the  model.     Curriculum  Development  is  on  the  left, 
Adoption  and  Diffusion  of  Innovations  is  on  the  right,  and 
Systems  theory  is  in  the  middle   (or  "back",   if  visualized 
three-dimensionally) .     Each  horizontal  level  of  the  model 
represents  a  frame  of  reference  drawn  from  the 
relationships  among  the  three  theories.     The  uppermost 
level  is  "Environment;"  the  second  is  "Structure;"  the 
third,   "Process;"  and  the  final  level  is  "Evaluation." 
These  four  levels  were  the  source  of  research  questions 
that  guided  the  descriptive  analysis  of  the  distance 
education  program. 

Some  examples  of  research  questions  that  grew  out  of 
the  "Environment"  level  of  the  model  included: 

What  are  Zimbabwe's  educational  needs? 

What  relationships  exist  between  the  program  and 

the  environment  ? 

•  What  are  the  goals  and  objectives  of  the 
distance  education  program? 

The  second  plane  of  reference  focused  on  the  distance 
education  program  and  its  content  and  internal  structure. 
Research  questions  included: 

•  What  components  constitute  the  distance 
education  program? 

•  What  are  the  functions  of  the  various 
components? 

•  Are  the  components  and  functions  working 
together  to  move  the  program  toward  its  goals? 

The  "Process"  level  of  the  model  was  concerned  with 
the  behavior  of  the  program  over  time  and  transformations 
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Figure  1.     Tr\ad  Perspective  Model  of  Distance 
Education  (TPMDE) 
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that  could  affect  the  overall  organization  of  program 
operations.     Some  research  questions  from  this  level  were: 

•  How  has  the  distance  education  program  changed 
over  time? 

How  could  purposeful  change  be  introduced  into 
the  system? 

The  final  level  of  the  TPMDE,   "Evaluation,"  is 
dependent  upon  the  communication  channels  available  among 
the  system's  components  and  also  those  supporting  the 
system  within  its  environmental  context.     The  monitoring 
and  rsgulatory  capacity  of  the  program  was  studied  by 
asking : 

•  What  communication  channels  are  necessary  for 
the  successful  functioning  of  the  program? 

•  Is  there  a  clearly  identified  process  for 
program  monitoring  within  the  distance  education 
program  structure? 

The  flexibility  of  this  integrated  model  allows  the 
researcher  to  generate  research  questions  related  to  a 
specific  program,   or  to  design  a  new  program  based  on 
identified  needs.     In  this  study,  a  comprehensive 
description  of  the  distance  education  component  of  the 
Zimbabwean  teacher  education  program  was  created.  The 
major  strengths  and  weaknesses  of  the  program  were 
identified  and  recommendations  for  further  development  were 
given . 

Strengths  and  Weakn^s.^  of  the  Distance  Education  Program 

There  were  five  major  strengths  identified  during  this 
study.     These  were:     purpose,  history.  Ministry  advocacy, 
personnel,  and  students. 

"Purpose"  refers  to  the  strength  of  the  conviction 
within  Zimbabwe's  educational  community  that  distance 
education  was  worthwhile  and  important  to  the  success  of 
the  teacher  education  program.     Initiated  in  response  to  a 
national  crisis  —  teacher  shortage  —  there  was  a  minimum 
of  resistance  to  this  innovation. 

The  second  strength  identified  was  the  program's 
persistence.     Faculty  members  in  the  teachers'  colleges 
were  comfortable  with  the  concept  of  distance  education  for 
instructional  delivery.     Distance  education  had  historical 
precedent  in  several  African  nations  and  was  considered  a 
viable  mode  of  instruction. 

Ministry  advocacy  of  the  distance  education  component 
was  the  third  program  strength.     As  the  administrative 
entity  over  the  teachers'   colleges.  Ministry  of  Education 
support  potentially  could  be  translated  into  budgetary  and 
personnel  assistance.     By  recognizing  the  distance 
education  component  as  a  vital  part  of  the  teacher 
education  program,  official  Ministry  sanction  also  provided 
political  endorsement  for  its  continued  existence. 
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The  fourth  strength  identified  was  the  quality  of 
personnel  involved  in  the  program.     The  theoretical 
grounding  and  academic  preparedness  of  these  educators 
created  a  strong  talent  pool  at  each  teachers'  college. 
Although  working  under  circumstances  that  might  appear 
unreasonable  in  the  United  States   (very  large  classes,  few 
resources,  etc.),   lecturers  spoke  positively  of  their  role 
in  the  educational  system. 

The  quality  of  the  students  at  the  teachers'  colleges 
was  the  fifth  major  program  strength.     Because  there  were 
many  more  applicants  than  program  openings,  the  colleges 
were  able  to  select  students  of  high  calibre  that  were  most 
likely  to  complete  the  program  successfully.  These 
students  were  described  by  their  instructors  as  highly 
motivated  and  dedicated,  and  those  encountered  by  the 
author  appeared  articulate  and  mature. 

The  primary  weaknesses  of  the  program  included  the 
lack  of  a  "Zimbabwean  Model"  for  distance  education, 
ineffective  communication,   lack  of  accountaoility, 
insufficient  quantities  of  trained  personnel,  and  lack  of 
instructor/student  feedback  procedures.     Each  of  these 
weaknesses  was  identified  by  using  the  TPMDE . 

The  lack  of  a  unified  model  for  distance  education  in 
the  Zimbabwean  teacher  education  program  contributed  to  a 
lack  of  focus.     Without  uniformity  of  purpose,  the  distance 
education  activities  across  colleges   (and,  in  some  cases, 
within  colleges)  were  designed  to  achieve  differing  aims. 
This,   in  turn,   led  to  inconsistency  in  programming  and  lack 
of  equivalency  across  programs. 

Lack  of  effective  communication  channels,  the  second 
major  weakness,   resulted  partially  from  the  lack  of  a 
unified  model.     Although  some  of  the  difficulty  in 
comm^^nicating  could  be  attributed  to  logistical  barriers 
(e.g.,  poor  telephone  and  postal  service),  much  of  the 
problem  could  be  attributed  to  the  system  itself.  Without 
an  established  structure  to  support  the  open  exchange  of 
ideas  and  information,  channels  of  communication  are 
unlikely  to  develop  spontaneously. 

The  third  weakness,   lack  of  accountability,  could  also 
be  related  to  the  lack  of  a  unified  structure  within  the 
program.     Because  the  distance  education  activities  at  each 
college  were  managed  by  individual  departments,   there  was 
no  centralized  system  in  place  to  monitor  the  overall 
progress  of  the  students.     This  made  it  difficult  to 
substantiate  the  value  of  the  program  and  undermined  the 
credibility  of  its  goals. 

Although  the  program's  personnel  was  considered  a 
major  strength,   there  were  simply  not  enough  of  them  —  a 
weakness  that  was  consistent  across  all  levels  of  the 
educational  system.     Lecturei:^  and  administrators  who  were 
on  the  job  became  overworked  and  were  unable  to  provide  as 
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rauch  individual  attention  to  students  as  they  would  have 
preferred . 

The  fifth  weakness  —  lack  of  timely  feedback  to 
students        had  severe  ramifications.     Distance  education 
projects  and  assignments  had  sometimes  remained  unmarked 
for  weeks   (or  even  months)   and  students  were  unaware  of 
their  progress.     Instructionally,  this  was  the  program's 
most  critical  flaw,  and  understaf f ing  and  logistical 
barriers  to  interaction  had  aggravated  the  problem. 

Program  Recommendations 

Based  on  the  comprehensive  description  ol  ^:he  distance 
education  component,  a  set  of  program  recomiTtendat ions  was 
developed.     These  were: 

•  Establish  a  Distance  Education  Commission; 

•  Conduct  program  evaluations  on  a  regular  basis; 

•  Re-focus  the  distance  education  modules  toward 
applied  education; 

•  Encourage  a  more  active  role  for  the  University 
of  Zimbabwe;  and 

•  Establish  a  Distance  Education  Department  at 
each  teachers'  college. 

Establishing  a  Distance  Education  Commission,  to  be 
convened  by  the  Ministry  of  Education,  would  enhance  the 
credibility  of  the  program  and  increase  the  communication 
among  the  parties  involved.     This  commission  would  serve 
predominantly  in  a  consultative  role  and  would  make 
recommendations  for  program  improvements  based  on  the 
commissioners '  collective  expertise . 

Conducting  program  evaluations  on  a  regularly- 
scheduled  basis  was  the  second  recommendation.  These 
rigorous,   objective  evaluations  could  examine  the  cost 
effectiveness  of  the  program,  attitudes  of  students, 
perceptions  of  lecturers  and  administrators,  quality  of 
materials,  and  student  achievement.     Collecting  program 
information  regularly  creates  a  base  of  research  data  that 
could  be  used  to  make  long-term  planning  decisions  c*nd  help 
to  allocate  limited  resources  ef  f  icienv.ly . 

The  third  recommendation  was  to  rr.odify  the  focus  of 
the  distance  education  modules,   emphasizing  applied 
education  topics  and  practical  aspects  of  teaching.  This 
would  enable  the  students  on  T.P.  to  utilize  their 
classrooms  for  assignments  directly  tied  to  their  daily 
activities.     Another  potential  benefit  of  this  modification 
would  be  improved  teaching  by  the  T.P.   students  and  a 
stronger  link  between  the  theoretical  topics  covered  in  on- 
campus  instruction  and  the  practical  realities  of  the 
classroom . 

Increasing  the  University  of  Zimbabwe's  involvement  in 
the  distance  education  program  was  the  fourth 
recommendation.     As  the  overseeing  institution  for  teacher 
certification,  a  stronger  leadership  position  was 
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suggested.     Some  possible  activities  related  to  this  would 
be  offering  courses  about  distance  education  for  college 
credit,  seeking  out  external  funding  sources  for  upgrading 
the  distance  education  program,   and  consulting  with 
individual  teachers'  colleges  regarding  research  projects. 

The  fifth  recommendation  was  to  establish  a  Distance 
Education  Department  at  each  teachers'*  college.  By 
centralizing  operations  in  this  way,  program  accountability 
would  be  facilitated  and  consistency  across  programs 
enhanced.     Communication  throughout  the  system  would  also 
be  improved  because  there  would  be  one  individual  at  each 
institution  that  could  "speak  for  the  program." 

Summary 

By  using  the  Triad  Perspective  Model  of  Distance 
Education  as  a  descriptive  tool/  the  distance  education 
component  of  the  teacher  education  program  of  Zimbabwe  was 
studied.     A  comprehensive  narrative  detailing  the  strengths 
and  weaknesses  of  the  distance  education  activities  was 
created  with  reconunendations  for  further  improvement.  As 
educational  access  becomes  increasingly  important  to 
traditional  educational  institutions,   the  role  of  distance 
education  will  expand.     To  this  end,  a  solid  research 
foundation  must  be  created  to  guide  decision-making  for 
long  term  success. 
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